ChatGPT is everywhere. Here’s where it came from

We’ve reached peak ChatGPT. Released in December as a web app by the San Francisco–based firm OpenAI, the chatbot exploded into the mainstream almost overnight. According to some estimates, it is the fastest-growing internet service ever, reaching 100 million users in January, just two months after launch. Through OpenAI’s $10 billion deal with Microsoft, the tech is now being built into Office software and the Bing search engine. Stung into action by its newly awakened onetime rival in the battle for search, Google is fast-tracking the rollout of its own chatbot, LaMDA. Even my family WhatsApp is filled with ChatGPT chat.

But OpenAI’s breakout hit did not come out of nowhere. The chatbot is the most polished iteration to date in a line of large language models going back years. This is how we got here.

1980s–’90s: Recurrent Neural Networks

ChatGPT is a version of GPT-3, a large language model also developed by OpenAI.  Language models are a type of neural network that has been trained on lots and lots of text. (Neural networks are software inspired by the way neurons in animal brains signal one another.) Because text is made up of sequences of letters and words of varying lengths, language models require a type of neural network that can make sense of that kind of data. Recurrent neural networks, invented in the 1980s, can handle sequences of words, but they are slow to train and can forget previous words in a sequence.

In 1997, computer scientists Sepp Hochreiter and Jürgen Schmidhuber fixed this by inventing LTSM (Long Short-Term Memory) networks, recurrent neural networks with special components that allowed past data in an input sequence to be retained for longer. LTSMs could handle strings of text several hundred words long, but their language skills were limited.  

2017: Transformers

The breakthrough behind today’s generation of large language models came when a team of Google researchers invented transformers, a kind of neural network that can track where each word or phrase appears in a sequence. The meaning of words often depends on the meaning of other words that come before or after. By tracking this contextual information, transformers can handle longer strings of text and capture the meanings of words more accurately. For example, “hot dog” means very different things in the sentences “Hot dogs should be given plenty of water” and “Hot dogs should be eaten with mustard.”

2018–2019: GPT and GPT-2

OpenAI’s first two large language models came just a few months apart. The company wants to develop multi-skilled, general-purpose AI and believes that large language models are a key step toward that goal. GPT (short for Generative Pre-trained Transformer) planted a flag, beating state-of-the-art benchmarks for natural-language processing at the time. 

GPT combined transformers with unsupervised learning, a way to train machine-learning models on data (in this case, lots and lots of text) that hasn’t been annotated beforehand. This lets the software figure out patterns in the data by itself, without having to be told what it’s looking at. Many previous successes in machine-learning had relied on supervised learning and annotated data, but labeling data by hand is slow work and thus limits the size of the data sets available for training.  

But it was GPT-2 that created the bigger buzz. OpenAI claimed to be so concerned people would use GPT-2 “to generate deceptive, biased, or abusive language” that it would not be releasing the full model. How times change.

2020: GPT-3

GPT-2 was impressive, but OpenAI’s follow-up, GPT-3, made jaws drop. Its ability to generate human-like text was a big leap forward. GPT-3 can answer questions, summarize documents, generate stories in different styles, translate between English, French, Spanish, and Japanese, and more. Its mimicry is uncanny.

One of the most remarkable takeaways is that GPT-3’s gains came from supersizing existing techniques rather than inventing new ones. GPT-3 has 175 billion parameters (the values in a network that get adjusted during training), compared with GPT-2’s 1.5 billion. It was also trained on a lot more data. 

But training on text taken from the internet brings new problems. GPT-3 soaked up much of the disinformation and prejudice it found online and reproduced it on demand. As OpenAI acknowledged: “Internet-trained models have internet-scale biases.”

December 2020: Toxic text and other problems

While OpenAI was wrestling with GPT-3’s biases, the rest of the tech world was facing a high-profile reckoning over the failure to curb toxic tendencies in AI. It’s no secret that large language models can spew out false—even hateful—text, but researchers have found that fixing the problem is not on the to-do list of most Big Tech firms. When Timnit Gebru, co-director of Google’s AI ethics team, coauthored a paper that highlighted the potential harms associated with large language models (including high computing costs), it was not welcomed by senior managers inside the company. In December 2020, Gebru was pushed out of her job.  

January 2022: InstructGPT

OpenAI tried to reduce the amount of misinformation and offensive text that GPT-3 produced by using reinforcement learning to train a version of the model on the preferences of human testers. The result, InstructGPT, was better at following the instructions of people using it—known as “alignment” in AI jargon—and produced less offensive language, less misinformation, and fewer mistakes overall. In short, InstructGPT is less of an asshole—unless it’s asked to be one.

May–July 2022: OPT, BLOOM

A common criticism of large language models is that the cost of training them makes it hard for all but the richest labs to build one. This raises concerns that such powerful AI is being built by small corporate teams behind closed doors, without proper scrutiny and without the input of a wider research community. In response, a handful of collaborative projects have developed large language models and released them for free to any researcher who wants to study—and improve—the technology. Meta built and gave away OPT, a reconstruction of GPT-3. And Hugging Face led a consortium of around 1,000 volunteer researchers to build and release BLOOM.      

December 2022: ChatGPT

Even OpenAI is blown away by how ChatGPT has been received. In the company’s first demo, which it gave me the day before ChatGPT was launched online, it was pitched as an incremental update to InstructGPT. Like that model, ChatGPT was trained using reinforcement learning on feedback from human testers who scored its performance as a fluid, accurate, and inoffensive interlocutor. In effect, OpenAI trained GPT-3 to master the game of conversation and invited everyone to come and play. Millions of us have been playing ever since.

The Download: inside our chaotic brains, and small nuclear reactors

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

Neuroscientists listened in on people’s brains for a week. They found order and chaos.

The news: Our brains exist in a state somewhere between stability and chaos as they help us make sense of the world, according to recordings of brain activity taken from volunteers over the course of a week. 

What it means: As we go from reading a book to chatting with a friend, for example, our brains shift from one semi-stable state to another—but only after chaotically zipping through multiple other states in a pattern that looks completely random.

Why it’s important: Understanding how our brains restore some degree of stability after chaos could help us work out how to treat disorders at either end of this spectrum. Too much chaos is probably what happens when a person has a seizure, whereas too much stability might leave a person comatose. Read the full story.

—Jessica Hamzelou

We were promised smaller nuclear reactors. Where are they?


For over a decade, we’ve heard that small reactors could be a big part of nuclear power’s future. In theory, small modular reactors (SMRs) could solve some of the major challenges of traditional nuclear power, making plants quicker and cheaper to build and safer to operate.  

Oregon-based NuScale recently became the first company of its kind to clear one of the final regulatory hurdles before the company can build its reactors in the US. But even as SMRs promise to speed up construction timelines for nuclear power, the path has been full of delays and cost hikes—and there’s still a whole lot of streamlining to do before they become commonplace. Read the full story.

—Casey Crownhart

How Telegram groups can be used by police to find protesters

Many Chinese individuals are still in police custody after going into the streets in Beijing late last year to protest zero-covid policies. While action happened in many Chinese cities, it’s the Beijing police who have been consistently making new arrests, as recently as mid-January. 

For the younger generations, the movement was an introduction to civil disobedience. But many people lack the technical knowledge to protect themselves when organizing or participating in public events—meaning that their digital communications could have left them open to being identified. Read the full story.

—Zeyi Yang

Zeyi’s story is from China Report, his weekly newsletter covering the country. Sign up to receive it in your inbox every Tuesday.

Podcast: The AI in the newsroom

OpenAI’s ChatGPT chatbot has taken the internet by storm since it launched late last year. The latest episode of our award-winning podcast, In Machines We Trust, delves into the benefits and potential pitfalls of using AI tools in newsrooms, and what it could mean for the future of journalism as we know it. Listen to it on Apple Podcasts, or wherever else you usually listen.

The must-reads

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 Microsoft has unveiled OpenAI-powered Bing
Tech companies are racing to revamp search engines with AI. (NYT $)
+ Some of Bing’s AI-boosted answers are a bit dodgy, though. (WP $)
+ Could this finally be a reason to use Bing? (Vox)

2 How the Chinese ‘spy balloon’ drama unfolded on TikTok 
With lots of silly jokes, and footage of the big “pop” moment. (WP $)
+ The US insists the balloon belonged to the Chinese military. (WP $)
+ What the balloon means for the latest iteration of the space race. (Vox)
+ A new cold war could be on the horizon. (Economist $)

3 Chipmakers are worried about a ‘forever chemicals’ ban
They’re concerned it’ll tip an already fragile industry over the edge. (FT $)
+ These simple design rules could turn the chip industry on its head. (MIT Technology Review)

4 We’re strengthening superbugs by destroying the environment
Antimicrobial resistance is on the rise, and so is environmental destruction. (Wired $)
+ We can use sewage to track the rise of antibiotic-resistant bacteria. (MIT Technology Review)

5 How Big Tech managed to water down the right to repair
Lobbyists successfully tweaked the US bill in phonemakers’ favor. (The Markup)

6 Digital payments aren’t taking off in Iraq
Decades of war and sanctions mean that citizens are still heavily reliant on cash. (Rest of World)
+ The country has just revalued its currency. (Reuters)

7 The problem with lab-grown meat
The experimental label isn’t a tasty incentive. (Bloomberg $)
+ Will lab-grown meat ever reach our plates? (MIT Technology Review)

8 Meet the human guinea pigs 
Innovators are increasingly experimenting on their own bodies. (Neo.Life)

9 We’re stopping ⚠️BeingReal⚠️
Downloads of the authenticity-prizing app are slumping. (Sifted)

10 Don’t expect any crypto ads at the Super Bowl 
The organizers have learnt their lesson. (Insider $)
+ Crypto exchange Binance has grown more powerful since FTX’s collapse. (FT $)
+ What’s next for crypto. (MIT Technology Review)

Quote of the day

“I would be hurt or offended if I found out my Valentine’s message was written by a machine / artificial intelligence.”

—A statement that 50% of polled people in the US agreed with, reports Fast Company.

The big story

What’s bigger than a megacity? China’s planned city clusters

April 2021

China has urbanized with unprecedented speed. About 20 years ago, only 30% of the Chinese population lived in cities; today it’s 60%. That translates to roughly 400 million people—more than the entire US population—moving into China’s cities in the past two decades.

To accommodate the influx, China’s national urban development policy has shifted from expanding individual cities to systematically building out massive city clusters. Cities in a cluster will collaborate economically, ecologically, and politically, the thinking goes, in turn boosting each region’s competitiveness. Read the full story.

—Ling Xin

We can still have nice things

A place for comfort, fun and distraction in these weird times. (Got any ideas? Drop me a line or tweet ’em at me.)

+ The legendary Sarah Michelle Gellar speaks!
+ Tech bros sure love their sick threads.
+ Hear me out: being grateful for the things we dislike can be an emotionally helpful exercise.  
+ I love this photo of Skin from Skunk Anansie accepting an award from King Charles.
+ Winter doesn’t have to be soul-destroying once Christmas is over. Here’s how to learn to love it.

How Telegram groups can be used by police to find protesters

China Report is MIT Technology Review’s newsletter about technology developments in China. Sign up to receive it in your inbox every Tuesday.

First of all, I’m still processing the whole “Chinese spy balloon” saga, which, from start to finish, took over everyone’s brains for just about 72 hours and has been one of the weirdest recent events in US-China relations. There are still so many mysteries around it that I don’t want to jump to any conclusions, but I will link to some helpful analyses in the next section. For now, I just want to say: RIP The Balloon.

On a wholly different note, I’ve been preoccupied by the many Chinese individuals who remain in police custody after going into the streets in Beijing late last year to protest zero-covid policies. While action happened in many Chinese cities, it’s the Beijing police who have been consistently making new arrests, as recently as mid-January. According to a Twitter account that’s been following what’s happened with the protesters, over 20 people have been detained in Beijing since December 18, four of them formally charged with the crime of “picking quarrels.” As the Wall Street Journal has reported, many of those arrested have been young women.

For the younger generation in China, the movement last year was an introduction to participating in civil disobedience. But many of these young people lack the technical knowledge to protect themselves when organizing or participating in public events. As the Chinese government’s surveillance capability grows, activists are forced to become tech experts to avoid being monitored. It’s an evolving lesson that every new activist will have to learn.

To better understand what has happened over the past two months and what lies ahead, I reached out to Lü Pin, a feminist activist and scholar currently based in the US. As one of the most prominent voices in China’s current feminist movement, Lü is still involved in activist efforts inside China and the longtime cat-and-mouse game between protesters and police. Even though their work is peaceful and legal, she and her fellow activists often worry that their communications are being intercepted by the government. When we talked last week about the aftermath of the “White Paper Protests,” she explained how she thinks protesters were potentially identified through their communications, why many Chinese protesters continue to use Telegram, and the different methods China’s traditional police force and state security agents use to infiltrate group chats.

The following interview has been translated, lightly edited, and rearranged for clarity.

How did the Chinese police figure out the identity of protesters and arrest them over a month after it happened?

In the beginning, the police likely got access to a Telegram group. Later on, officers could have used facial recognition [to identify people in video footage]. Many people, when participating in the White Paper Protests, were filmed with their faces visible. It’s possible that the police are now working on identifying more faces in these videos.

Those who were arrested have no way of confirming this, but their friends [suspect that facial recognition was used] and spread the message. 

And, as you said, it was reported that the police did have information on some protesters’ involvement in a Telegram group. What exactly happened there?

When [these protesters in Beijing] decided to use a Telegram group, they didn’t realize they needed to protect the information on the event. Their Telegram group became very public in the end. Some of them even screenshotted it and posted it on their WeChat timelines. 

Even when they were on the streets in Liangma River [where the November 27 protest in Beijing took place], this group chat was still active. What could easily have happened was that when the police arrested them, they didn’t have time to delete the group chat from their phone. If that happened, nothing [about the group] would be secure anymore.

Could there be undercover police in the Telegram group?

It’s inevitable that there were government people in the Telegram group. When we were organizing the feminist movement inside China, there were always state security officials [in the group]. They would use fake identities to talk to organizers and say: I’m a student interested in feminism. I want to attend your event, join your WeChat group, and know when’s the next gathering. They joined countless WeChat groups to monitor the events. It’s not just limited to feminist activists. They are going to join every group chat about civil society groups, no matter if you are [advocating for] LGBTQ rights or environmental protection. 

What do they want to achieve by infiltrating these group chats?

Different Chinese ministries have different jobs. The people collecting information [undercover] are mostly from the Ministry of State Security [Editor’s note: this is the agency responsible for foreign intelligence and counterintelligence work]. It operates on a long-term basis, so it would be doing more information collection; it has no responsibility to call off an event.

But the purpose of the Ministry of Public Security [Editor’s note: this is the rank-and-file police force] is to stop our events immediately. It works on a more short-term basis. According to my experience, the technology know-how of the police is relatively [basic]. They mostly work with WeChat and don’t use any VPN. And they are also only responsible for one locality, so it’s easier to tell who they are. For example, if they work for the city of Guangzhuo, they will only care about what’s going to happen in Guangzhou. And people may realize who they are because of that.

I’m also seeing people question whether some Twitter accounts, like the one belonging to “Teacher Li,” were undercover police. Is there any merit to that thinking?

It used to be less complicated. Previously, the government could use censorship mechanisms to control [what people posted] within China, so they didn’t need to [establish phishing accounts on foreign platforms]. But one characteristic of the White Paper Revolution is that it leveraged foreign platforms more than ever before.

But my personal opinion is that the chance of a public [Twitter] account phishing information for the government is relatively small. The government operations don’t necessarily have intricate planning. When we talk about phishing, we are talking about setting up an account, accepting user submissions, monitoring your submissions remotely, and then monitoring your activities. It requires a lot of investment to operate a [public] account. It’s far less efficient than infiltrating a WeChat group or Telegram group to obtain information.

But I don’t think the anxiety is unwarranted. The government’s tools evolve rapidly. Every time the government has learned about our organizing or the information of our members, we try to analyze how it happened. It used to be that we could often find out why, but now we can hardly figure out how the police found us. It means their data investigation skills have modernized. So I think the suspicion [of phishing accounts’ existence] is understandable.

And there is a dilemma here: On one hand, we need to be alert. On the other hand, if we are consumed by fears, the Chinese government will have won. That’s the situation we are in today.

When did people start to use Telegram instead of WeChat?

I started around 2014 or 2015. In 2015, we organized some rescue operations [for five feminist activists detained by the state] through Telegram. Before that, people didn’t realize WeChat was not secure. [Editor’s note: WeChat messages are not end-to-end encrypted and have been used by the police for prosecution.] Afterwards, when people were looking for a secure messaging app, the first option was Telegram. At the time, it was both secure and accessible in China. Later, Telegram was blocked, but the habit [of using it] remained. But I don’t use Telegram now.

It does feel like Telegram has gained this reputation of “the protest app of choice” even though it’s not necessarily the most secure one. Why is that?

If you are just a small underground circle, there are a lot of software options you can use. But if you also want other people to join your group, then it has to be something people already know and use widely. That’s how Telegram became the choice. 

But in my opinion, if you are already getting out of the Great Firewall, you can use Signal, or you can use WhatsApp. But many Chinese people don’t know about WhatsApp, so they choose to stay on Telegram. It has a lot to do with the reputation of Telegram. There’s a user stickiness issue with any software you use. Every time you migrate to new software, you will lose a great number of users. That’s a serious problem.

So what apps are you using now to communicate with protesters in China?

The app we use now? That’s a secret [laughs]. The reason why Telegram was monitored and blocked in the first place was because there was lots of media reporting on Telegram use back in 2015.

What do you think about the security protocols taken by Telegram and other communication apps? Let me know at zeyi@technologyreview.com.

Catch up with China

1. The balloon fiasco caused US Secretary of State Antony Blinken to postpone his meeting with President Xi Jinping of China, which was originally planned for this week. (CNN)

  • While the specific goals of the balloon’s trip are unclear, an expert said the termination mechanism likely failed to function. (Ars Technica)
  • Since the balloon was shot down over the weekend, the US Coast Guard has been searching for debris in the Atlantic, which US officials hope to use to reconstruct Chinese intelligence-gathering methods. (Reuters $)
  • The balloon itself didn’t necessarily pose many risks, but the way the situation escalated makes clear that military officials in the two countries do not currently have good communication. (New York Times $

2. TikTok finally opened a transparency center in LA, three years after it first announced it’d build new sites where people could examine how the app conducts moderation. A Forbes journalist who was allowed to tour the center wasn’t impressed. (Forbes)

3. Baidu, China’s leading search engine and AI company, is planning to release its own version of ChatGPT in March. (Bloomberg $)

4. The past three months should have been the busiest season for Foxconn’s iPhone assembly factory in China. Instead, it was disrupted by mass covid-19 infections and intense labor protests. (Rest of World)

5. A new decentralized social media platform called Damus had its five minutes (actually, two days) of fame in China before Apple swiftly removed it from China’s App Store for violating domestic cybersecurity laws. (South China Morning Post $)

6. Taiwan decided to shut down all nuclear power plants by 2025. But its renewable-energy industry is not ready to fill in the gap, and now new fossil-fuel plants are being built to secure the energy supply. (HuffPost)

7. The US Department of Justice suspects that executives of the San Diego–based self-driving-truck company TuSimple have improperly transferred technology to China, anonymous sources said. (Wall Street Journal $)

Lost in translation

Renting smartphones is becoming a popular alternative to purchasing them in China, according to the Chinese publication Shenran Caijing. With 19 billion RMB ($2.79 billion) spent on smartphone rentals in 2021, it is a niche but growing market in the country. Many people opt for rentals to be able to brag about having the latest model, or as a temporary solution when, for example, their phone breaks down and the new iPhone doesn’t come out for a few months. 

But this isn’t exactly saving people cash. While renting a phone costs only one or two bucks a day, the fees build up over time, and many platforms require leases to be at least six months long. In the end, it may not be as cost-effective as buying a phone outright. 

The high costs and lack of regulation have led some individuals to exploit the system. Some people use it as a form of cash loan: they rent a high-end phone, immediately sell it for cash, and slowly pay back the rental and buyout fees. There are also cases of scams where people use someone else’s identity to rent a phone, only to disappear once they obtain the device.

One more thing

Born in Wuhan, I grew up eating freshwater fish like Prussian carp. They taste divine, but the popular kinds often have more small bones than saltwater fish, which can make the eating experience laborious and annoying. Last week, a team of Chinese hydrobiologists based in Wuhan (duh) announced that they had used CRISPR-Cas9 gene-editing technology to create a Prussian carp mutant that is free of the small bones. Not gonna lie, this is true innovation to me.

CT scans from the academic paper showing the original fish and the mutant version without small bones.

We were promised smaller nuclear reactors. Where are they?

For over a decade, we’ve heard that small reactors could be a big part of nuclear power’s future.

Because of their size, small modular reactors (SMRs) could solve some of the major challenges of traditional nuclear power, making plants quicker and cheaper to build and safer to operate.  

That future may have just gotten a little closer. In the past month, Oregon-based NuScale has reached several major milestones for its planned SMRs, most recently receiving a final approval from the US federal government for its reactor design. Other companies, including Kairos Power and GE Hitachi Nuclear Energy, are also pursuing commercial SMRs, but NuScale’s reactor is the first to reach this stage, clearing one of the final regulatory hurdles before the company can build its reactors in the US. 

SMRs like NuScale’s planned reactors could provide power when and where it’s needed in easy-to-build, easy-to-manage plants. The technology could help curb climate change by replacing plants powered by fossil fuels, including coal.

But even as SMRs promise to speed up construction timelines for nuclear power, the path to this point has been full of delays and cost hikes. And the road ahead for NuScale still stretches years into the future, revealing just how much streamlining there still is to go before this form of nuclear power could be built quickly and efficiently.

Going smaller

NuScale’s SMR generates electricity by a process similar to the one used in today’s nuclear plants: the reactor splits atoms in a pressurized core, giving off heat. That heat can be used to turn water into steam that powers a turbine, generating electricity. The biggest difference is the size of the reactors.

In the past, nuclear plants have been gigantic undertakings—so-called megaprojects, costing billions of dollars. “If it’s over a billion dollars, the wheels tend to fall off on a project,” says Patrick White, a project manager at the Nuclear Innovation Alliance, a nuclear-focused think tank.

For example, construction is currently underway in Georgia to install two additional units at the existing Vogtle power plant. Each of the two planned units will have a capacity of over 1,000 megawatts, enough to power over a million homes. The reactors were supposed to start up in 2017. They still haven’t, and the project’s total cost has doubled, to over $30 billion, since construction began a decade ago.

By contrast, NuScale plans to build reactor modules that have a capacity of less than 100 megawatts. When these modules are combined in power plants, they’ll add up to a few hundred megawatts, smaller than even a single unit in the Vogtle plant. SMR plants with a capacity of a few hundred megawatts would power several hundred thousand homes—similar to an average-size coal-fired power plant in the US. 

And while the Vogtle plant sits on a site that covers more than 3,000 acres, NuScale’s SMR project should require about 65 acres of land. 

Smaller nuclear power facilities could be easier to build and might help cut costs as companies standardize designs for reactors. “That’s the benefit—it becomes more of a routine, more of a cookie-cutter project,” says Jacopo Buongiorno, director of the Center for Advanced Nuclear Energy Systems at MIT.

These reactors might also be safer, since the systems needed to keep them cool, as well as those needed to shut them down in an emergency, could be simpler. 

Untangling the red tape

The problem with all these potential benefits is that so far, they’re still mostly potential. Demonstration projects have started up in some parts of the world, with China being the first to connect an SMR to the electrical grid in 2021. Last month, GE Hitachi Nuclear Energy signed commercial contracts for a plant in Ontario, which could come online in the mid-2030s. NuScale, too, is pursuing projects in Romania and Poland. 

There are no SMRs running in the US yet, partly because of the lengthy regulatory process run by the Nuclear Regulatory Commission (NRC), an independent federal agency.

Nuclear is the only power source to have its own dedicated regulatory agency in the US. That extra oversight means no detail goes unnoticed, and it can take a while to get nuclear projects moving. “These are big, complicated projects,” says Kathryn Huff, assistant secretary in the office of nuclear energy at the US Department of Energy. The DOE helps fund SMR projects and support research, but it doesn’t oversee nuclear regulations.

NuScale started working toward regulatory approval in 2008 and submitted its official application to the NRC in 2016. In 2020, when it received a design approval for its reactor, the company said the regulatory process had cost half a billion dollars, and that it had provided about 2 million pages of supporting documents to the NRC.

After more than two years of finalizing details and a vote by the agency, the NRC released its final ruling on NuScale’s reactor design last month. The final ruling goes into effect on February 21 and certifies a NuScale design for a reactor module that generates 50 MW of electricity.

Receiving a final ruling for the design means that NuScale would only have to get approval for a reactor site and complete final safety reviews before beginning construction. So in theory, NuScale has already cleared the hardest regulatory steps required before building a reactor.

“It is a big deal and should be celebrated as a milestone,” Buongiorno says. However, he says, minimizing what’s still to come would be a mistake: “Nothing is easy and nothing is quick when it comes to the NRC.”

There’s an additional wrinkle: NuScale wants to tweak its reactor modules. While the company was going through the lengthy regulatory process, researchers were still working on reactor design. During the process of submission and planning, the company discovered that its reactors could achieve better performance.

“We found that we could actually produce more power with the same reactor, the same exact size,” says Jose Reyes, cofounder and chief technology officer at NuScale. Instead of 50 MW, the company found that each module could produce 77 MW. 

So the company changed course. For its first power plant, which will be built at the Idaho National Laboratory, NuScale is planning to package six of the higher-capacity reactors together, making the plant capacity 462 MW in total.

The upgraded power rating requires some adjustments, but the module design is fundamentally the same. Still, it means that the company needed to resubmit updated plans to the NRC, which it did last month. It could take up to two years before the altered plans are approved by the agency and the company can move on to site approval, Reyes says.

The long road ahead

Back in 2017, NuScale planned to have its first power plant in Idaho running and generating electricity for the grid by 2026. That timeline has been pushed back to 2029. 

Meanwhile, costs are higher than when the regulatory process first kicked off. In January, NuScale announced that its planned price of electricity from the Idaho plant project had increased, from $58 per megawatt-hour to $89. That’s more expensive than most other sources of electricity today, including solar and wind power and most natural-gas plants. 

The price hikes would be even higher if not for substantial federal investment. The Department of Energy has already pitched in over $1 billion to the project, and the Inflation Reduction Act passed last year includes $30/MWh in credits for nuclear power plants.

Costs have gone up for many large construction projects, as inflation has affected the price of steel and other building materials while interest rates have risen. But the increases also illustrate what often happens with first-of-their-kind engineering projects, Buongiorno says: companies may try to promise quick results and cheap power, but “these initial units will always be a little bit behind schedule and a little bit above budget.”

If price hikes continue, there’s a chance that participants could back out of NuScale’s project, which could spell danger. For SMRs in the works, “I’m not going to believe it’s for real until I see them operating,” Buongiorno says. 

The true promise of SMRs will be realized only when it’s time to build the second, the third, the fifth, and the hundredth reactor, DOE’s Huff says, and both companies and regulators are learning how to speed up the process to get there. But the benefits of SMRs are all theoretical until reactors are running, supplying electricity without the need for fossil fuels.

“It becomes truly real when electrons go on the grid,” Huff says.

Neuroscientists listened in on people’s brains for a week. They found order and chaos.

Our brains exist in a state somewhere between stability and chaos as they help us make sense of the world, according to recordings of brain activity taken from volunteers over the course of a week. As we go from reading a book to chatting with a friend, for example, our brains shift from one semi-stable state to another—but only after chaotically zipping through multiple other states in a pattern that looks completely random.

Understanding how our brains restore some degree of stability after chaos could help us work out how to treat disorders at either end of this spectrum. Too much chaos is probably what happens when a person has a seizure, whereas too much stability might leave a person comatose, say the neuroscientists behind the work. 

A better understanding of what’s going on could one day allow us to use brain stimulation to tip the brain into a sweet spot between the extremes.

A week in the brain

Brain imaging techniques have revealed a lot about how the brain works—but there’s only so much you can learn by getting a person to lie still in a brain scanner for half an hour. Avniel Ghuman and Maxwell Wang at the University of Pittsburgh wanted to know what happens in the longer term. After all, the symptoms of many neurological disorders can develop over hours or days, says Wang. To get a better idea of what might be going on, the pair devised an experiment that would let them watch brain activity for around a week.

Ghuman, Wang, and their colleagues turned to people who were undergoing brain surgery for epilepsy. Some people with severe or otherwise untreatable epilepsy opt to have the small parts of their brain that trigger their seizures surgically removed. Before any operation, they may have electrodes implanted in their brains for a week or so. During that time, these electrodes monitor brain activity to help surgeons pinpoint where their seizures start and identify exactly which bit of brain should be removed.

The researchers recruited 20 such individuals to volunteer in their study. Each person had 10 to 15 electrodes implanted for somewhere between three and 12 days.

The pair collected recordings from the electrodes over the entire period. The volunteers were all in hospital while they were monitored, but they still did everyday things like eating meals, talking to friends, watching TV, or reading books. “We know so little about what the brain does during these real, natural behaviors in a real-world setting,” says Ghuman.

The edge of chaos

The team found some surprising patterns in brain activity over the course of the week. Specific brain networks seemed to communicate with each other in what looked like a “dance,” with one region appearing to “listen” while the other “spoke,” say the researchers, who presented their findings at the Society for Neuroscience annual meeting in San Diego last year.

And while the volunteers’ brains seemed to pass between different states over time, they did so in a curious way. Rather than simply moving from one pattern of activity to another, their brains appeared to zip between several other states in between, apparently at random. As the brain shifts from one semi-stable state to another, it seems to embrace chaos.

It makes sense, says Rick Adams, a psychiatrist and neuroscientist at University College London, who was not involved in the work. “There’s probably no central node that tells the rest of the brain what to do,” he says. “It’s a bit like shaking a snow globe—you introduce some random variation and trust that if it goes through a bunch of configurations, the optimal one will pop out somehow.”

“There are stable states, and then there are unpredictable, volatile transitions,” says Hayriye Cagnan, a neuroscientist at the University of Oxford, who was not involved in the research. If we can figure out the pattern associated with a healthy brain, we might be able to use electrical stimulation to treat neurological disorders, she says.

That’s what Ghuman hopes. Healthy patterns of brain activity are “somewhere on the edge of order and disorder,” he says. “This may be an optimal place for the brain to be.”

The results don’t yet tell us what a healthy brain functioning in a natural environment might look like. After all, all the volunteers were in the hospital, waiting for brain surgery to treat their severe seizures. But the team hopes that their study provides the first step to figuring this out.

The approach could help us develop better treatments for epilepsy, too. Some people opt to have electrodes implanted in their brains that sense when a seizure is starting and deliver a pulse of electricity to head them off. These devices aren’t perfect, though. They might work better if they were developed to recognize these chaotic transitions and nudge the brain into a place between chaos and stability, suggests Kelly Bijanki, a neuroscientist at Baylor College of Medicine in Houston, Texas.

In the future, Ghuman and Wang hope to use the same approach to find out what happens in children’s brains and whether it differs from the activity seen in adults. They also hope to learn more about how our brains change over the course of a day or a week, and how this is linked to our body’s circadian rhythms.

The Download: generative AI for video, and detecting AI text

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

The original startup behind Stable Diffusion has launched a generative AI for video

What’s happened: Runway, the generative AI startup that co-created last year’s breakout text-to-image model Stable Diffusion, has released an AI model that can transform existing videos into new ones by applying styles from a text prompt or reference image.

What it does: In a demo reel posted on its website, Runway shows how the model, called Gen-1, can turn people on a street into claymation puppets, and books stacked on a table into a cityscape at night. Other recent text-to-video models can generate very short video clips from scratch, but because Gen-1adapts existing footage it can produce much longer videos.

Why it matters: Last year’s explosion in generative AI was fueled by the millions of people who got their hands on powerful creative tools for the first time and shared what they made, and Runway hopes Gen-1 will have a similar effect on generated videos. Read the full story.

—Will Douglas Heaven

Why detecting AI-generated text is so difficult (and what to do about it)

Last week, OpenAI unveiled a tool that can detect text produced by its AI system ChatGPT. But if you’re a teacher who fears the coming deluge of ChatGPT-generated essays, don’t get too excited.

The tool is still very much a work in progress, and it is woefully unreliable, only identifying 26% of AI-written text correctly as “likely AI-written” 26% of the time. Even while we should expect this number to improve, we’re extremely unlikely to ever get a tool that can spot AI-generated text with 100% certainty. Read the full story.

—Melissa Heikkilä

Melissa’s story is from The Algorithm, her weekly AI newsletter. Sign up to receive it in your inbox every Monday.

The must-reads

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 Google has announced its own chatbot to rival ChatGPT
Bard won’t be available to the public for another few weeks. (The Verge)
+ It’s called Bard because it’s a storyteller, geddit? (NYT $)
+ AI is kindling the long-dormant search wars flame. (FT $)
+ Only ‘trusted testers’ have access to the system right now. (Wired $)

2 China’s ‘spy balloon’ furore has undermined Xi Jinping’s leadership
It certainly isn’t helping his ongoing efforts to stabilize tense relations with the US. (FT $)
+ The balloon was 200ft tall and carrying a huge load. (BBC)

3 We can’t really predict earthquakes
Even the best efforts aren’t able to provide much more than a few seconds’ warning.(WP $)
+ Rescuers in Turkey and Syria are working around the clock to find survivors. (FT $)
+ A deep-learning algorithm could detect earthquakes by filtering out city noise. (MIT Technology Review)

4 Andrew Tate groomed women into joining his webcam sex ring
Victims were approached on social media and dating apps. (Vice)
+ The “manosphere” is getting more toxic as angry men join the incels. (MIT Technology Review)

5 What it’s like to run an abortion hotline 
Post-Roe, its volunteers are dealing with more calls than ever. (Vox) + The cognitive dissonance of watching the end of Roe unfold online. (MIT Technology Review)

6 This viral TikTok drug challenge never actually existed
But that hasn’t stopped panic from spreading across Mexico. (Rest of World)
+ The porcelain challenge didn’t need to be real to get views. (MIT Technology Review)

7 We’re still learning about the moon 🌒
NASA ShadowCam mission hopes to shed some light on its most mysterious craters. (The Atlantic $)

8 Austin has become a refuge for tech workers left jaded by Silicon Valley
The city’s cool, counter-culture vibe is a massive draw.(New Yorker $)

9 How EV batteries are made in America
But some components still need to be sourced from overseas. (WSJ $)+ How old batteries will help power tomorrow’s EVs. (MIT Technology Review)

10 The tricky ethics of de-aging actors with AI
Some critics argue it’s both demeaning and pointless. (The Guardian)
+ AI has learnt how to crush humans at Pokémon. (The Atlantic $)

Quote of the day

“People are afraid to have conversations.” 

—Mary Jane Copps, a former journalist who coaches people on how to speak to others over the phone, tells Bloomberg why her customers are so reluctant to talk. 

The big story

How big technology systems are slowing innovation

February 2022

In 2005, years before Apple’s Siri and Amazon’s Alexa came on the scene, two startups—ScanSoft and Nuance Communications—merged to pursue a burgeoning opportunity in speech recognition. The new company developed powerful speech-processing software and grew rapidly for almost a decade. Then suddenly, around 2014, it stopped growing.

Nuance’s story is far from unique. In all major industries and technology domains, startups are facing unprecedented obstacles. They are growing much more slowly than comparable companies did in the past. And it will take not only strong antitrust enforcement to reverse the trend, but a fundamental loosening of restrictions like non-compete agreements and intellectual property rights. Read the full story.

—James Bessen

We can still have nice things

A place for comfort, fun and distraction in these weird times. (Got any ideas? Drop me a line or tweet ’em at me.)

+ These paintings (and the odd photo) of London are magical.
+ Now this is a chatbot I can get onboard with—Cat GPT (thanks Charlotte! Top tip: meow at it)
+ If you’re worried about your attention span, you should feel safe in the knowledge that even ancient Roman philosophers struggled to concentrate sometimes.
+ Happy quarter-century to that sinister dancing baby.

Why detecting AI-generated text is so difficult (and what to do about it)

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Last week, OpenAI unveiled a tool that can detect text produced by its AI system ChatGPT. But if you’re a teacher who fears the coming deluge of ChatGPT-generated essays, don’t get the party poppers out yet. 

This tool is OpenAI’s response to the heat it’s gotten from educators, journalists, and others for launching ChatGPT without any ways to detect text it has generated. However, it is still very much a work in progress, and it is woefully unreliable. OpenAI says its AI text detector correctly identifies 26% of AI-written text as “likely AI-written.” 

While OpenAI clearly has a lot more work to do to refine its tool, there’s a limit to just how good it can make it. We’re extremely unlikely to ever get a tool that can spot AI-generated text with 100% certainty. It’s really hard to detect AI-generated text because the whole point of AI language models is to generate fluent and human-seeming text, and the model is mimicking text created by humans, says Muhammad Abdul-Mageed, a professor who oversees research in natural-language processing and machine learning at the University of British Columbia

We are in an arms race to build detection methods that can match the latest, most powerful models, Abdul-Mageed adds. New AI language models are more powerful and better at generating even more fluent language, which quickly makes our existing detection tool kit outdated. 

OpenAI built its detector by creating a whole new AI language model akin to ChatGPT that is specifically trained to detect outputs from models like itself. Although details are sparse, the company apparently trained the model with examples of AI-generated text and examples of human-generated text, and then asked it to spot the AI-generated text. We asked for more information, but OpenAI did not respond. 

Last month, I wrote about another method for detecting text generated by an AI: watermarks. These act as a sort of secret signal in AI-produced text that allows computer programs to detect it as such. 

Researchers at the University of Maryland have developed a neat way of applying watermarks to text generated by AI language models, and they have made it freely available. These watermarks would allow us to tell with almost complete certainty when AI-generated text has been used. 

The trouble is that this method requires AI companies to embed watermarking in their chatbots right from the start. OpenAI is developing these systems but has yet to roll them out in any of its products. Why the delay? One reason might be that it’s not always desirable to have AI-generated text watermarked. 

One of the most promising ways ChatGPT could be integrated into products is as a tool to help people write emails or as an enhanced spell-checker in a word processor. That’s not exactly cheating. But watermarking all AI-generated text would automatically flag these outputs and could lead to wrongful accusations.

The AI text detector that OpenAI rolled out is only one tool among many, and in the future we will likely have to use a combination of them to identify AI-generated text. Another new tool, called GPTZero, measures how random text passages are. AI-generated text uses more of the same words, while people write with more variation. As with diagnoses from doctors, says Abdul-Mageed, when using AI detection tools it’s a good idea to get a second or even a third opinion.

One of the biggest changes ushered in by ChatGPT might be the shift in how we evaluate written text. In the future, maybe students won’t write everything from scratch anymore, and the focus will be on coming up with original thoughts, says Sebastian Raschka, an AI researcher who works at AI startup Lightning.AI. Essays and texts generated by ChatGPT will eventually start resembling each other as the AI system runs out of ideas, because it is constrained by its programming and the data in its training set.

“It will be easier to write correctly, but it won’t be easier to write originally,” Raschka says.

New report: Generative AI in industrial design and engineering

Generative AI—the hottest technology this year—is transforming entire sectors, from journalism and drug design to industrial design and engineering. It’ll be more important than ever for leaders in those industries to stay ahead. We’ve got you covered. A new research report from MIT Technology Review highlights the opportunities—and potential pitfalls— of this new technology for industrial design and engineering. 

The report includes two case studies from leading industrial and engineering companies that are already applying generative AI to their work—and a ton of takeaways and best practices from industry leaders. It is available now for $195.

Deeper Learning

AI models generate copyrighted images and photos of real people

Popular image generation models such as Stable Diffusion can be prompted to produce identifiable photos of real people, potentially threatening their privacy, according to new research. The work also shows that these AI systems can be made to regurgitate exact copies of medical images, as well as copyrighted work by artists. 

Why this matters: The extent to which these AI models memorize and regurgitate images from their databases is at the root of multiple lawsuits between AI companies and artists. This finding could strengthen the artists’ case. Read more from me about this

Leaky AI models: Sadly, in the push to release new models faster, AI developers too often overlook privacy. And it’s not just image-generating systems. AI language models are also extremely leaky, as I found out when I asked GPT-3, ChatGPT’s predecessor, what it knew about me and MIT Technology Review’s editor in chief. The results were hilarious and creepy.  

Bits and Bytes

When my dad was sick, I started Googling grief. Then I couldn’t escape it.
A beautiful piece by my colleague Tate Ryan-Mosley about grief and death, and the pernicious content recommendation algorithms that follow her around the internet only to offer more content on grief and death. Tate spent months asking experts how we can get more control over rogue algorithms. Their answers aren’t all that satisfying. (MIT Technology Review)

Google has invested $300 million into an AI startup 
The tech giant is the latest to hop on the generative-AI bandwagon. It’s poured money into AI startup Anthropic, which is developing language models similar to ChatGPT. The deal gives Google a 10% stake in the company in exchange for the computing power needed to run large AI models. (The Financial Times)

How ChatGPT kicked off an AI race
This is a nice peek behind the scenes at OpenAI and how they decided to launch ChatGPT as a way to gather feedback for the next-generation AI language model, GPT-4. The chatbot’s success has been an “earthshaking surprise” inside OpenAI. (The New York Times

If ChatGPT were a cat
Meet CatGPT. Frankly, the only AI chatbot that matters to me.

The original startup behind Stable Diffusion has launched a generative AI for video

Runway, the generative AI startup that co-created last year’s breakout text-to-image model Stable Diffusion, has released an AI model that can transform existing videos into new ones by applying any style specified by a text prompt or reference image.

In a demo reel posted on its website, Runway shows how its software, called Gen-1, can turn clips of people on a street into claymation puppets, or books stacked on a table into a cityscape at night. Runway hopes that Gen-1 will do for video what Stable Diffusion did for images. “We’ve seen a big explosion in image-generation models,” says Runway CEO and cofounder Cristóbal Valenzuela. “I truly believe that 2023 is going to be the year of video.”

Set up in 2018, Runway has been developing AI-powered video-editing software for several years. Its tools are used by TikTokers and YouTubers as well as mainstream movie and TV studios. The makers of The Late Show with Stephen Colbert used Runway software to edit the show’s graphics; the visual effects team behind the hit movie Everything Everywhere All at Once used the company’s tech to help create certain scenes.  

In 2021, Runway collaborated with researchers at the University of Munich to build the first version of Stable Diffusion. Stability AI, a UK-based startup, then stepped in to pay the computing costs required to train the model on much more data. In 2022, Stability AI took Stable Diffusion mainstream, transforming it from a research project into a global phenomenon. 

But the two companies no longer collaborate. Getty is now taking legal action against Stability AI—claiming that the company used Getty’s images, which appear in Stable Diffusion’s training data, without permission—and Runway is keen to keep its distance.

RUNWAY

Gen-1 represents a new start for Runway. It follows a smattering of text-to-video models revealed late last year, including Make-a-Video from Meta and Phenaki from Google, both of which can generate very short video clips from scratch. It is also similar to Dreamix, a generative AI from Google revealed last week, which can create new videos from existing ones by applying specified styles. But at least judging from Runway’s demo reel, Gen-1 appears to be a step up in video quality. Because it transforms existing footage, it can also produce much longer videos than most previous models. (The company says it will post technical details about Gen-1 on its website in the next few days.)   

Unlike Meta and Google, Runway has built its model with customers in mind. “This is one of the first models to be developed really closely with a community of video makers,” says Valenzuela. “It comes with years of insight about how filmmakers and VFX editors actually work on post-production.”

Gen-1, which runs on the cloud via Runway’s website, is being made available to a handful of invited users today and will be launched to everyone on the waitlist in a few weeks.

Last year’s explosion in generative AI was fueled by the millions of people who got their hands on powerful creative tools for the first time and shared what they made with them. Valenzuela hopes that putting Gen-1 into the hands of creative professionals will soon have a similar impact on video.

“We’re really close to having full feature films being generated,” he says. “We’re close to a place where most of the content you’ll see online will be generated.”