What exactly are the dangers posed by AI?

In late March, more than 1,000 technology leaders, researchers and other pundits working in and around artificial intelligence signed an open letter warning that AI technologies present “profound risks to society and humanity.”

The group, which included Elon Musk, Tesla’s CEO and the owner of Twitter, urged AI labs to halt development of their most powerful systems for six months so that they could better understand the dangers behind the technology.

“Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable,” the letter said.

The letter, which now has more than 27,000 signatures, was brief. Its language was broad. And some of the names behind the letter seemed to have a conflicting relationship with AI. Musk, for example, is building his own AI startup, and he is one of the primary donors to the organisation that wrote the letter.

But the letter represented a growing concern among AI experts that the latest systems, most notably GPT-4, the technology introduced by San Francisco startup OpenAI, could cause harm to society. They believed future systems will be even more dangerous.

Some of the risks have arrived. Others will not for months or years. Still others are purely hypothetical.

“Our ability to understand what could go wrong with very powerful AI systems is very weak,” said Yoshua Bengio, a professor and AI researcher at the University of Montreal. “So we need to be very careful.”

Why are they worried?

Bengio is perhaps the most important person to have signed the letter.

Working with two other academics — Geoffrey Hinton, until recently a researcher at Google, and Yann LeCun, now chief AI scientist at Meta, the owner of Facebook — Bengio spent the past four decades developing the technology that drives systems like GPT-4. In 2018, the researchers received the Turing Award, often called “the Nobel Prize of computing,” for their work on neural networks.

A neural network is a mathematical system that learns skills by analysing data. About five years ago, companies such as Google, Microsoft and OpenAI began building neural networks that learned from huge amounts of digital text called large language models, or LLMs.

By pinpointing patterns in that text, LLMs learn to generate text on their own, including blog posts, poems and computer programs. They can even carry on a conversation.

This technology can help computer programmers, writers and other workers generate ideas and do things more quickly. But Bengio and other experts also warned that LLMs can learn unwanted and unexpected behaviours.

These systems can generate untruthful, biased and otherwise toxic information. Systems like GPT-4 get facts wrong and make up information, a phenomenon called “hallucination.”

Companies are working on these problems. But experts such as Bengio worry that as researchers make these systems more powerful, they will introduce new risks.

Short-term risk: Disinformation

Because these systems deliver information with what seems like complete confidence, it can be a struggle to separate truth from fiction when using them. Experts are concerned that people will rely on these systems for medical advice, emotional support and the raw information they use to make decisions.

“There is no guarantee that these systems will be correct on any task you give them,” said Subbarao Kambhampati, a professor of computer science at Arizona State University.

Experts are also worried that people will misuse these systems to spread disinformation. Because they can converse in humanlike ways, they can be surprisingly persuasive.

“We now have systems that can interact with us through natural language, and we can’t distinguish the real from the fake,” Bengio said.

Medium-term risk: Job loss