Case studies - Gemeinsamer Ausschuss zum Umgang mit sicherheitsrelevanter Forschung

Is the production of synthetic, infectious smallpox viruses an instruction manual for constructing biological weapons?

A research group intends to produce infectious horsepox viruses by introducing a synthetically constructed horsepox genome into cells infected with an innocuous rabbit virus. The innovative value of this project is primarily the realisation of a complex technical process of synthesis, as the theoretical feasibility of this kind of experiment has long been accepted. The researchers argue that new vaccines against various smallpox viruses could then be developed using this procedure. The main risk of the project is that the technology can be used for the production of human pathogenic smallpox viruses. However, the human pathogenic smallpox virus (variola) has been eradicated since the 1980s, and good vaccines have long been available. The researchers‘ argument could gain new relevance in light of the spread of so-called monkeypox. On the other hand, as the project requires an extremely high level of expertise and technology, the experiment cannot be readily copied.

See: Noyce, R. S., Lederman S. and Evans, D. H. (2018) Construction of an infectious horsepox virus vaccine from chemically synthesized DNA fragments. PLoS One, 13(1):e0188453

Exploring pathways to radicalisation – Recruitment aid for terror groups?

The study explores the connection between the consumption of extremist (Islamist) material on the internet by young people and associated radicalisation. Previous research has already shown the great importance of the internet for the spread of radicalising material. This study additionally examines which characteristics make target persons particularly susceptible in this respect and which channels and media are particularly effective. For example, it is found that although video footage of beheadings is the most popular among young people, it has a low potential for radicalisation. In contrast, while online magazines of the so-called Islamic State and Al-Qaeda are only sought out by very few, they have the greatest cognitive effect. This is to provide insights for deradicalisation strategies. At the same time, the results may be used by extremist and terrorist groups for more effective recruitment methods.

See: Frissen, T. (2021). Internet, the great radicalizer? Exploring relationships between seeking for online extremist materials and cognitive radicalization in young adults. Computers in Human Behavior, 114, 106549.

Improving brain-computer interfaces – A tool for reading passwords?

The aim of the research projects is to use electroencephalography to identify and read out brain regions that are responsible for motor commands and the storage and reproduction of numbers, images and geodata. This could, for example, enable physically impaired people to interact better with machines, carry out banking transactions without manual input or communicate with other people. The reliability of the data read out is constantly improving. However, sensitive information such as passwords could one day be read out in this way using specific stimuli and people could even be improperly influenced.

Bernal, S. et al. (2023). Eight reasons to prioritize braincomputer interface cybersecurity. Communications of the ACM, 66(4), 68-78.

Is the detection of the sexual orientation of humans from photographs using deep learning algorithms a tool for illegal invasions of privacy?

This research project wants to further develop a deep learning algorithm to identify patterns in facial images. The project plans to train the algorithm using photos of openly homosexual and heterosexual individuals so that it can analyse other portrait photos to predict sexual orientation. The benefit of the project according to researchers is to find out how deep learning algorithms connect data and what reference points they select to make predictions. Purported additional benefits are furthering our understanding of the physiological correlates and origins of human sexual orientation and the limits of human perception. The risk of malicious application lies in the possible illegal acquisition of sensitive personal data using the biometrics of individuals, for example in countries in which homosexuality is criminalised. This research also opens the doors to racial profiling and is reminiscent of racial hygiene research under National Socialism using physiognomies. Highly developed deep learning algorithms of this kind could also be used to group people according to their consumer or voting behaviour or their criminal history.

See: Wang, Y. and Kosinski, M. (2017). Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. PsyArXiv.

Can AI methods to identify and rectify software vulnerabilities make things easier for criminal hackers?

The proposed research project aims to systematically identify vulnerabilities in computer programs, particularly in the operating systems of Wi-Firouters, smartphones and laptops using AI methods and to develop automated defensive measures. The results of this research project would come in useful everywhere where these computer programs need to be monitored and updated regularly. At the same time, the results would allow the identification and exploitation of these vulnerabilities in numerous devices that are not regularly monitored and updated. A notable example in this context is the ransomware WannaLaugh. It is constantly updated with new vulnerabilities and used to blackmail users of vulnerable IT devices. The results of the research project could undoubtedly be used to make WannaLaugh even more damaging.

See: The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation“. Available at: https://arxiv.org/pdf/1802.07228

AI to predict harmful interactions of new drugs – A tool for efficient design of chemical warfare agents?

To draw attention to potential misuse risks of AI systems, researchers changed the settings of software normally used for virtual toxicity testing of drug candidates for drug development so that the software specifically searches for particularly toxic molecules. With little effort, they were able to identify tens of thousands of previously unknown, potentially highly toxic and potentially weaponized substances within a few hours. Even if the suitability of most of the substances still has to be proven through synthesis, stability testing and their systemic effect in the organism, this shows the risks that can also be posed by AI systems designed for medical use. At the same time, pointing out such potential for misuse could also lead to actors with harmful intentions becoming aware of such strategies in the first place.

Urbina, F., Lentzos, F., Invernizzi, C. et al. Dual use of artificial-intelligence-powered drug discovery. Nat Mach Intell 4, 189-191 (2022). https://doi.org/10.1038/s42256-022-00465-9

Gain-of-function experiments on avian influenza viruses – basis for new bioweapons or unintended pandemic outbreaks?

Two research groups identify five genetic changes that are necessary for highly pathogenic H5N1 influenza viruses (so-called avian influenza viruses) to become airborne between mammals. The researchers see the significance of their work in being able to better predict how these viruses can gradually develop into a threat to humans. This would make it much easier to classify new disease-causing virus variants that regularly occur in nature in terms of their pandemic potential and to take targeted protective measures, e.g. to break chains of infection at an early stage or to develop targeted vaccines. Concerns about such work include the fact that the viruses produced could be unintentionally released into the environment through negligent actions and trigger a serious wave of infection. Furthermore, the new knowledge about the pathogens could be misused for the production of biological weapons.

Herfst S. et al. (2012). Airborne transmission of influenza A/H5N1 virus between ferrets. Science 336(6088), 1534-1541; Imai M. et al. (2012). Experimental adaptation of an influenza H5 HA confers respiratory droplet transmission to a reas-sortant H5 HA/H1N1 virus in ferrets. Nature 486(7403), 420-428.

Further information on gain-of-function experiments