Recent Peer-Reviewed Publications:

  • “Extractive versus Generative Language Models for Political Conflict Text Classification.” with Patrick T. Brandt, Sultan Alsarra, Vito J. D’Orazio, Latifur Khan, Shreyas Meher, Javier Osorio, & Marcus Sianan (2025). Forthcoming at Political Analysis.

    Abstract:

    We review our recent ConfliBERT language model (Hu et al. 2022) to process political and violence related texts. When fine-tuned, results show that ConfliBERT has superior performance in accuracy, precision and recall over other large language models (LLMs) like Google’s Gemma 2 (9B), Meta’s Llama 3.1 (7B), and Alibaba’s Qwen 2.5 (14B) within its relevant domains. It is also hundreds of times faster than these more generalist LLMs. These results are illustrated using texts from the BBC, re3d, and the Global Terrorism Database (GTD). We demonstrate that open, fine-tuned models can outperform the more general models in terms of accuracy, precision, recall and at a fraction of the cost.

  • “Unpacking gendered co-participation in political violence: Women perpetrators of the 1994 genocide in Rwanda.” with Jared F. Edgerton, Elizabeth L. Brannon, & Hollie Nyseth Nzitatira (2025). American Journal of Political Science

    Abstract:

    How does gender influence participation in violence? Research shows that women are less likely than men to engage in direct violence. However, women remain consequential actors in conflicts. Drawing on gendered understandings of conflict, we argue that mobilization is shaped by gendered homophily within social networks. We theorize that both men and women are likely to mobilize with individuals of the same gender. However, this effect is more pronounced for women due to differences in how men and women are mobilized for conflict, as well as other forms of political engagement. To test this argument, we utilize data from the 1994 genocide that targeted Tutsi in Rwanda. Using network analysis, we demonstrate the pivotal role women played in mobilizing other women to commit violence. This article broadens our understanding of network dynamics in conflict and emphasizes the importance of gendered differences in mobilization patterns for political processes.

Recent Conference Publications:

  • “The Devil is in the Details: Assessing the Effects of Machine-Translation on LLM Performance in Domain-Specific Texts.” with Javier Osorio, Afraa Alshammari, Naif Alatrush, Dagmar Heintze, Amber Converse, Sultan Alsarra, Latifur Khan, Patrick T Brandt & Vito D’Orazio (2025). Proceedings of Machine Translation Summit XX: Volume 1

    Abstract:

    Conflict scholars increasingly use computational tools to track violence and cooperation at a global scale. To study foreign locations, researchers often use machine translation (MT) tools, but rarely evaluate the quality of the MT output or its effects on Large Language Model (LLM) performance. Using a domain-specific multi-lingual parallel corpus, this study evaluates the quality of several MT tools for text in English, Arabic, and Spanish. Using ConfliBERT, a domain-specific LLM, the study evaluates the effect of MT texts on model performance, and finds that MT texts tend to yield better results than native texts. The MT quality assessment reveals considerable translation-induced distortions, reductions in vocabulary size and text specialization, and changes in syntactical structure. Regression analysis at the sentence-level reveals that such distortions, particularly reductions in general and domain vocabulary rarity, artificially boost LLM performance by simplifying the MT output. This finding cautions researchers and practitioners about uncritically relying on MT tools without considering MT-induced data loss.

  • “Keep it local: Comparing domain-specific LLMs in native and machine translated text using parallel corpora on political conflict.” with Javier Osorio, Sultan Alsarra, Amber Converse, Afraa Alshammari, Latifur Khan, NaifAlatrush, Patrick T Brandt, Vito D’Orazio, Niamat Zawad & Mahrusa Billah (2024). 2024 2nd International Conference on Foundation and Large Language Models (FLLM)

    Abstract:

    The dynamics of political conflict and cooperation require powerful computerized tools capable of effectively tracking security threats and cooperation around the world. This study compares the performance of domain-specific Large Language Models (LLMs) against generically-trained LLMs in binary and multi-class classification using native text in English, Spanish, and Arabic, and their corresponding machine translations. This endeavor yields four key contributions. 1) We present and make available a novel database of annotations using a multi-lingual parallel corpus from the United Nations. 2) Using various metrics, we assess the quality of different machine translation tools. 3) Our results indicate that the ConfliBERT family of LLMs, a set of domain-specific models tailored for political conflict, outperform generically-trained LLMs in English, Spanish, and Arabic in both binary and multi-class tasks. 4) We also disentangle the heterogeneous effects of machine translation on LLM performance in different languages. Overall, results reveal the comparative advantage of native-language domain-specific LLMs specialized on political conflict to understand the dynamics of violence and cooperation worldwide using native text. Our multi-lingual ConfliBERT LLMs provide critical cyber-infrastructure enabling scholars and government agencies use their local languages and information to foster safer, more stable political environments.