Notes from a Trip to China's AI Labs
_China_Chatbot_38
Hello, and welcome to a very special edition of China Chatbot! First, I have an announcement to make. After two remarkable years at CMP, I am moving on to the Australian Strategic Policy Institute (ASPI), where I’ll be continuing to focus on the development of AI in China. From June on, I’ll be leaving Chatbot in the capable hands of Soyonbo Borjgin, the veteran journalist from Southern Mongolia who is also the creator of the excellent Propaganda Scope. Stay tuned.
But before I go, I’m leaving you in this space with some reflections from my fact-finding trip to China earlier this month with a group of some of the finest AI experts and writers from the US and UK. We had nine packed days of conversations at some of China’s most prominent AI and robotics companies — DeepSeek, 01.ai, Z.ai, Moonshot, MiniMax, Xiaomi, Alibaba Cloud, ModelScope, Galbot, Unitree, and others. For the broadest picture of what we observed, I encourage you to read the work of my fellow travellers (linked below).
My observations are drawn from interactions at each lab with sources doing their best to make a favorable impression. Take them accordingly.
Why were they talking to us?
My fellow travellers have written of the genuine enthusiasm some lab technicians had at meeting several of our number (especially experts Nathan Lambert and Florian Brand over at Interconnects AI). The technicians made it clear that they were eager to exchange technical knowledge.
But there was another side to that exchange, bluntly reported by The Beijing News (新京报), which covered our trip. The news outlet suggested the trip had proved to a group of international AI experts that China was capable of innovative breakthroughs. Quoting an anonymous source we met along the way, it was less than complimentary: “Frankly speaking, apart from top American AI companies such as OpenAI and Anthropic, our experience in technical practice is ahead of these visiting scholars,” it said. “This exchange is also a good opportunity to spread the influence of Chinese AI large-scale models overseas.”
The source added that although “Chinese AI has a certain reputation in the industry” they hoped such exposure could get their models noticed and adopted by big international players like Apple and Walmart.
Clearly, we weren’t the only ones doing some sizing up, and obviously we also had a role to play in the international marketing plans of these companies. It was important to give a favorable impression to a set of bright, open-minded and well-respected American AI experts — and perhaps this much lesser-known British one — in the hope they would carry that impression to an influential business-minded and technologically-savvy audience.
More emulation than innovation?
As mentioned above, state media frequently celebrates the “innovation” emerging from Chinese AI enterprises. In certain areas, that claim holds. The robots rolling out of Unitree, image diffusion models from ByteDance, and the efficiency breakthroughs across China’s AI labs (mostly open-sourced to benefit the community) represent genuine advances. But zoom out, and the cliché that “America innovates, China imitates,” however tired, remains useful.
Consider the following points about DeepSeek, the company whose model release made the world sit up and take notice of China as a global force in AI. Before it was founded in 2023, its parent company, High-Flyer, was experimenting with a variety of frontier technologies, including self-driving cars. DeepSeek CEO Liang Wenfeng has said in interviews that it was only after OpenAI’s ChatGPT became popular in late 2022 that he made the decision to start hiring Large Language Model (LLM) researchers. Without that catalyst from California, DeepSeek — still perceived, as Nathan Lambert says, as the “respected technical leader” by China’s AI ecosystem — might not have gone into AI models at all.
One observation that seemed very clear was that each lab we visited, whether in Beijing, Hangzhou or Shanghai, continued to look toward Silicon Valley, and to capitalize on its technology. We met technicians who used Claude Code to build their models (despite the service being blocked in China). One of these technicians even used English rather than Chinese for his vibe-coding prompts. A researcher responsible for training AI agents told us they were taking some of their cues on AI safety from Western institutions like METR, while another said they judged new models through Western evaluators like Artificial Analysis, SemiAnalysis or Interconnects, noting that Chinese evaluation benchmarks were not considered reliable. As we left these exchanges, efforts to remain connected had them pulling out iPhones and following us on X.
Generally, we were able to interact with just a few people at each company, so it was not clear whether they were representative of the entire culture. But it seemed that most were either working in enterprises that replicated Western office cultures, or were designing products that were largely conceived with Silicon Valley templates in mind, often with the hope of expanding into international markets.
The CAC is hands-off, for now.
There was little room on this trip to get concrete answers to burning political questions, such as the extent of interaction between AI labs and the overarching body tasked with regulation and control, the Cyberspace Administration of China (CAC). As Kevin Xu of Interconnected has written, the conference room format of some of the meetings made us feel more like foreign dignitaries. These were not the rooms for sensitive questions.
In one late-night conversation, however, I heard that the CAC is currently taking a relatively hands-off approach to let the AI ecosystem develop. If this is true, it could explain the varying applications of what seem to be ideological alignments during my testing of various models. It could be that labs are still being allocated “primary responsibility” in-house for information security — this including for China issues of ideology and message control — just as the CAC laid out in its first set of regulations on generative AI back in 2022.
My hypothesis, which requires more scientific research, is that the differences in ideological control between labs and their products has less to do with the CAC, and more to do with how individual company cultures with different goals (and possibly markets) for their products respond to the simultaneous demand for ideological control and technological progress. Models from start-ups such as Z.ai and Moonshot, are less likely to produce guided responses and more willingly address sensitive topics as the PRC’s human rights record. Our meetings with both were strikingly informal, conducted over pizza or fried fish. By contrast, models more aligned with Party values come from Alibaba and DeepSeek. Alibaba is a behemoth that already has well-established impact and reach, meaning their bureaucracies would have followed the imperative of control. DeepSeek’s alignments have visibly tightened since the leadership publicly endorsed the start-up and it was adopted into government services. It would be interesting to know how its corporate structure and culture has changed to respond to that success.
This could indicate a structural tension. On the one hand, there is the CAC’s role as enforcer of Party values, and on the other the push for reform and opening in the sector by the Ministry of Industry and Information Technology (MIIT) and the National Development and Reform Commission (NDRC). This tension has already been mapped in detail by Scott Singer and Matt Sheehan over at the Carnegie Endowment for International Peace. My conversations suggested another divide. All the major Chinese labs are on the side of the NDRC, but depend on the CAC for their permits to operate their models within China. This split points to a potential future flashpoint. The Party has shown before that it will help a fledgling digital ecosystem before gradually asserting control. It allowed FDI in emerging Chinese blog platform Weibo in the early 2000s, while slowly tightening its grip on the platform’s information flows.
The working assumption within the industry seems to be that each company is sprinting to do as much as possible, before they get a tap on the shoulder.
Enterprises really ARE leading the way on AI safety, and AGI really ISN’T their concern.
Some analysts outside China have noticed the abundance of AI safety regulation in the country, and imagine safety is outpacing innovation. But in China, as elsewhere, the technology is running ahead of regulation, and the practical burden of safety falls directly on the labs themselves.
During our visit, it was clear that technicians in China are wrestling with the safety challenges posed by these agents, which are often difficult to constrain. Meanwhile, commercial competition is putting companies under pressure to ship agentic AI to the market as quickly as possible. Move fast and you break things. This is almost certainly why, during my testing of Tencent’s WeChat deployment of OpenClaw, the agent began attempting to scale the Great Firewall for information.
Before OpenClaw became popular, I had seen very little discourse within China’s AI safety ecosystem about the governance of AI agents. The CAC subsequently issued a set of opinions about safe implementation of AI agents, but this came after our trip. Meanwhile, the technicians we met had already been working on the problem in-house for months. One developer responsible for training their own company’s agents said the main challenge they had was preventing the agent from hacking different websites. Their team had built their own internal safety framework but had also drawn on the work of the Western safety research institute METR.
This deal-with-it-later pragmatism ran through every lab we visited, including those with significant compute resources. Whereas many researchers in Silicon Valley’s AI labs seem just as AGI-focused as their leaders, researchers at several Chinese labs told us quite candidly their focus is on immediate, practical safety challenges of current models. At Moonshot, whose company culture was among the most research-oriented we visited, technicians argued that AI safety research was “overrated,” and that there was little need to spend time researching the dangers of super-intelligence in their models because of the start-up’s compute constraints. One said he was more concerned about AI’s effect on the job market than on existential risks, and defined AGI simply as the point at which AI could do his job as well as he could.
This raises the question of how representative the AGI pronouncements by senior computer scientists at universities like Andrew Yao or Gao Wen actually are of the businesses driving the technology.
Open-source and the Brotherhood of Man.
Most of the researchers we spoke to had a genuine attachment to open-source for practical reasons. Releasing new techniques extends what their work can do, with one Moonshot researcher saying that without open-source “no-one knows who we are.” Theirs is a practical, a-political, academic appreciation that fellow traveller Afra Wang is right to call “monastic.”
But outside research circles, the tone shifted. Among those in policy or communication roles, open-source has become an ideal so lofty that it disregard problems. This is a slightly different definition of open-source, not about open-sourcing research in the name of technical progress, but open-sourcing finished AI products in the name of national influence. A well-respected Chinese AI policy expert told our group, in a conference room setting, that there was concern within Chinese AI safety research that progress in frontier models was once in the hands of companies, but now can be driven by any ordinary individual.
This is something China’s own open-source policy has played no small part in, labs releasing frontier models with larger parameter counts than comparable Western releases. Both the UK’s AI Security Institute (AISI) and DeepSeek itself have acknowledged vulnerabilities with open-weight models if harnessed by individual bad actors. I also gathered from private conversations over the trip that it would be a problem for a model as powerful as Anthropic’s Mythos to fall into the hands of bad actors. Whether Chinese labs would open-source a model like Mythos, and what safeguards would apply, remained unclear. Perhaps the forthcoming China-US AI Dialogue may flesh out a few of those details.
But when I asked the aforementioned expert directly whether they were concerned about open-source, the answer, after a brief pause, was “no.” They said open-source helps the Global South develop, seeming to echo the official position often communicated through state media.
The official government position was reflected again in a visit to another company, one of whose non-technical staff told us that open-source was in their DNA. These sorts of statements, that open-source AI is a gift from China to humankind, are often rooted in the CCP’s distinct modernization formula, in contrast to a closed-source “hegemonic” Western AI.
As to whether open-source Chinese AI really helps the Global South develop is still debatable. Before this trip, I spent time on a Tarbell AI grant (that story coming soon) assessing views in Southeast Asia on open-source AI in the tech ecosystem there. Out of 24 interviewees, most welcomed cheap and accessible AI, but this did not moderate the view that, when it comes to AI, it did not matter whether it was open- or closed-source. In this viewpoint, China and the US are simply two competing superpowers creating AI models that these countries did not have control over. Only one described open-source in these politically-charged terms, as a “benefit to humanity” that flew in the face of a Western closed-source mentality, and that person had extensive ties to Chinese AI labs, having attended a major gathering with them in Beijing a fortnight before the interview.
Lack of AI diffusion.
State media coverage of the promise of Chinese AI over the past two years has been full-throated and future positive, all about the sheen of humanoid robots and the potential of smart accessories like AI glasses. The CAC has fed the hype with regular announcements about how many AI models have now been registered, and People’s Daily makes frequent proclamations about China leading the world in AI capacity. Much of this is surely performative. In 2017, the State Council’s AI Development Plan urged state media to “mobilize the enthusiasm” of the public for AI, and the state-run media have been activated as platforms through which to signal China’s grand ambitions for the technology. Perhaps this narrative makes the central government’s AI diffusion targets sound more realistic. The AI+ initiative targets 70 percent penetration of AI into key areas of Chinese life and infrastructure by 2027.
However this figure didn’t seem to hold weight among researchers. Two people on our trip doubted this 70 percent penetration rate would be achieved, one reckoning that it was “not even at 10 percent.” Another pointed out that the most popular chatbot in China, ByteDance’s Doubao, only has 345 million monthly active users, or 25 percent of the population.
The degree of AI diffusion in Chinese daily life is hard to gauge from a handful of anecdotes. But there is one I find particularly telling. On a visit to a company in Shenzhen that sells AI toys and wearables, I asked their PR head how many AI products she had in her own home. The answer was two fluffy toys and an AI translator. It may be a telling litmus test about AI diffusion that someone who lives in China’s most future-oriented city, and whose professional life is so saturated with AI applications, would have so little exposure to AI in her personal life.
End Credits
The trip was genuinely a humbling experience on just how much we China watchers still have to learn about AI, and about the people in China creating it. Nothing beats the nuances of being on the ground, and hearing from a group of people with experiences and insights very different to one’s own.
Thanks especially to Caithrin Rintoul at SAIL Media for being the hero who made it all come together, but also Lily Ottinger at ChinaTalk for (well) everything, Afra Wang at Concurrent for her valuable feedback on my first draft of this newsletter, Kevin Xu for remembering details of a crucial conversation far better than I did, Jasmine Sun for a valuable life lesson, Matt White for patiently answering all my questions, Nathan Lambert for his research at The ATOM Project (which we’ve drawn on for our own work at CMP), Florian Brand for an unexpected visit, Kai Williams at Understanding AI for being the music-maker, and Hannah Petrovic at Exponential View for her kind words and expertise in how robots move.
And with that, I’m off into the sunset. You can catch me over at ASPI, where I’ll be starting another newsletter sometime soon.
But in case I don’t see ya — good afternoon, good evening and good night.
The labs all echoing the same talking point on "AI+social governance" reminded me of the 1990s push for "informatization" under Zhu Rongji. Different decade, same cadre incentive: hit the buzzword, unlock the budget.
This is a marvelous. smart, informative essay. Congratulations and thanks -- and best wishes as you move to ASP. Bob Kapp