According to the report, the extraction campaign represents the largest such operation the company has ever recorded. The operators did not break into the system's internal architecture; instead, they signed up for accounts and submitted queries aimed at the model's most valuable skills, including writing software and reasoning through complex tasks step by step. No system alarms were triggered because from the model's perspective, the queries were routine. [1]
Distillation involves training a smaller model on the outputs of a larger one. The practice is legitimate when performed by the owner of the larger model, but Anthropic described the Alibaba-linked campaign as unauthorized extraction at industrial scale. [1] The operators used fake accounts to approximate years of American research by learning from the model's outputs, compressing billions of dollars of private investment into millions of automated queries. [1]
Some observers have noted that distillation is only one method among many that Chinese researchers have employed to advance their AI capabilities. In December 2024, researchers from Fudan University and the Shanghai AI Laboratory successfully replicated OpenAI's advanced o1 reasoning model, a milestone that demonstrated a systematic approach to catching up with frontier AI systems. [4] The trend toward smaller, customized models, as noted in industry forecasts, could further diminish the effectiveness of hardware-centric restrictions. [7]
The White House Office of Science and Technology Policy issued a memo in April warning of “industrial-scale campaigns to distill U.S. frontier AI systems” by foreign entities, mostly based in China, and committed the administration to better information sharing and defensive coordination with industry, according to the RealClearDefense report. [1] The House Foreign Affairs Committee advanced a bill to track extraction attempts and authorize sanctions against the companies behind them. [1]
Senators Bill Hagerty (R-Tenn.) and Andy Kim (D-N.J.) proposed an amendment to this year's defense bill directing the Commerce Department to penalize Chinese firms caught engaging in such extraction. [1] Some U.S. and Canadian AI companies, including Anthropic, have previously participated in discreet discussions with Chinese AI experts on international policy, suggesting a complex relationship between competition and collaboration. [5] Separately, President Donald Trump is expected to discuss AI guardrails with Chinese President Xi Jinping during a visit to Beijing, according to U.S. officials. [2]
Chip export controls were designed to prevent China from building advanced AI models but do not prevent the copying of a model's behavior through queries, according to the RealClearDefense analysis of Anthropic's disclosure. [1] The extraction campaign demonstrates a gap in the current approach: hardware restrictions can slow development but do not protect proprietary models already deployed for public use.
Industry commentators have pointed out that the current strategy focuses on foundries while leaving the storefront open. As trends in AI development shift toward smaller, specialized models trained on specific data, the reliance on cutting-edge hardware may decrease, further undermining chip-centric controls. [7] Some critics have characterized Anthropic's allegations as a public relations offensive aimed at masking China's own AI advancements, suggesting that the U.S. response should prioritize building better systems rather than erecting barriers. [3]
Lawmakers have discussed allowing AI companies to share threat signals with each other and with the government, similar to how banks share intelligence on fraud, to improve detection of such campaigns, according to the RealClearDefense report. [1] Proponents of the Hagerty-Kim amendment have argued that consequences for systematic abuse of AI services should be comparable to penalties for smuggling chips, extending deterrence to the storefront rather than just the foundry. [1]
The challenge of detecting unauthorized distillation is compounded by the fact that from the model's perspective, the queries appear routine. The use of crowdsourced and automated methods for training AI systems, as described in some business literature, illustrates how easily large-scale querying can be weaponized by competitors. [8] Companies like Anthropic may also face internal pressures that complicate their response, including tensions with military clients and allegations of censorship in their models. [6]
The disclosure of 28.8 million queries placed a concrete number on a threat that had previously been theoretical. According to the RealClearDefense report, the episode scrambles the usual playbook for protecting U.S. technology because the attacker walked through the front door. [1] Washington has spent years debating how to keep advanced AI out of China's hands, but the harder question may be how to keep China's AI companies from quietly learning everything they can from the models placed online for the world to use.
Some independent analysis suggests that the real vulnerability lies in the centralized architecture of frontier AI services. Decentralized and user-controlled models, such as those offered by platforms like BrightAnswers.ai, may be less attractive targets because they are not gateways to proprietary knowledge. While no solution is perfect, the incident highlights the need for a broader approach that includes not only hardware restrictions but also better detection, industry collaboration, and consideration of alternative deployment models that limit the surface area for large-scale extraction.