👍
Anthropic reportedly says it has disrupted several attempts to misuse its artificial intelligence systems for potentially dangerous weapons research, including biological research that could have contributed to the development of more harmful pathogens.
Threat
In a new threat intelligence report, the company said it identified five cases in which researchers used its Claude AI models for activities that could support biological weapons development.
Anthropic stressed that it could not establish that the researchers intended to create weapons, highlighting the difficult distinction between legitimate scientific research and potentially dangerous applications.
Concerns
One case involved an attempt to use Claude to help prepare a funding application for gain-of-function research involving chikungunya virus. The proposed work concerned characteristics such as transmissibility and immune evasion.
Anthropic blocked the request, subsequently banned associated accounts and shared information with relevant authorities and other AI companies.
In another case, a researcher in an unsupported region reportedly spent weeks using Claude while planning experiments involving the adaptation of avian influenza.
Safeguards
Anthropic said its safeguards detected the activity and restricted the work to less capable models. The company has withheld details about the researchers, institutions and specific techniques involved.
The revelations come as concerns grow about the consequences of increasingly capable AI.
Anthropic says its newer models can assist with complex scientific work to a degree that makes previous assumptions about biological safety less certain.
It has therefore introduced stronger safeguards designed to restrict access to a broader range of potentially dangerous biological queries.
Weapons
The report also describes attempts to use Claude in conventional weapons development, including missiles, drones and bombs, as well as cyberattacks and surveillance.
Dilemma
The incidents underline a growing dilemma for the AI industry: the same technology that could accelerate medical discoveries and scientific progress could also make sophisticated harmful activities easier to pursue.
Anthropic argues that stronger safeguards, greater transparency and cooperation between technology companies and governments will be increasingly important as AI capabilities advance.