AI for good sounds great. Does it work?
ACEIR director Prof. Murray Leibbrandt with J-PAL's Sam Carter and Sarah Kopper on the third day of the ARUA biennial conference in Kampala, Uganda.
The 2025 ARUA biennial conference featured two presentations from the Abdul Latif Jameel Poverty Action Lab (J-PAL), a global research centre at the Massachusetts Institute of Technology that aims to contribute to poverty reduction through policy informed by rigorous, scientific evidence.
The first was a keynote by J-PAL’s Senior AI Policy Manager, Samantha Carter, who discussed harnessing innovation and AI to address inequalities and poverty.
Sam's keynote was followed by the workshop that introduced participants to randomised controlled trials (RCTs) as a rigorous method for assessing the impact of AI tools, using real-world case studies from Kenya and Ghana.
After a packed three days of presentations focusing largely on the potential of AI across a swathe of sectors – including health, employment, education, and climate research – the workshop aimed to encourage participants to think critically about the outcomes their AI tools aimed to improve, and to build in a way to evaluate the success of improving these outcomes from the outset of product design and rollout.
The workshop began with an introduction to the concept of causal impact and the importance of a counterfactual group that is comparable to the group receiving an intervention in order to evaluate a programme’s effectiveness. Through a case study on a WhatsApp-based AI mentor for Kenyan entrepreneurs, participants learned about key considerations in designing a randomised evaluation, such as selecting the appropriate unit of randomisation, sampling and sample size considerations, and threats to an evaluation’s integrity.
Participants also reflected on considerations specific to AI-based interventions, such as those around data privacy, algorithmic bias, and inclusivity. These considerations were illustrated in the case study where, despite entrepreneurs reporting the AI tool as useful, the RCTs revealed nuanced results: high-performing businesses benefited, but lower-skilled entrepreneurs were worse off.
The approximately 30 workshop participants comprised a wide range of backgrounds, from economics and data science to public health and law. While some participants had used RCTs in their own research, most were new to this methodology. Several attendees expressed interest in applying these principles to their own research, particularly in the education and health sectors.
For more information on J-PAL's work, visit the website.