THOMAS SO – NOVEMBER 25TH, 2019 EDITOR: ANDY BABSON
In the past three decades, microeconomists have been relying on two main methods for analyzing the public’s well-being, and the cause and effects of various events on the economy. One method—natural experiments—is conducted when economists look into a real-life event that has naturally separated out two groups of subjects because one group was affected, while the other was not (e.g. a study on how a rise in minimum wages in New Jersey affected its employment level compared to its neighboring state Pennsylvania). However, not all topics of interest come along with convenient real life cases. As a result, researchers often rely on the second main method, which is having to create their own experiments using randomized controlled trials (RCTs). This method usually involves grouping people or communities living in a similar environment and giving “treatment” to one of the groups, while leaving the other as a control. Through this method, economists can study the differences observed between the two groups after the “treatment,” and develop them into economic arguments on the causes and effects of such a “treatment.” This method has been lauded as the “gold standard” of policy research and has been frequently used by developmental economists to analyze which policies are most efficient at alleviating poverty.
Recently, RCTs have come into the spotlight, as the Nobel Prize in Economics was awarded to three development economists, Abhijit Banerjee and Esther Duflo of M.I.T., and Michael Kremer of Harvard, for their work using such research methods to analyze small-scale interventions that are targeted to help the world’s most impoverished populations. One of the examples included a study on mosquito nets led by Michael Kremer and his colleagues in Kenya, in which he found that even a small fee for buying nets could dramatically reduce the amount of people who will purchase and use them, increasing the risk of diseases like malaria, thus endangering the health of neighborhoods. This result was used to refute the argument that a small charge ensures that people who purchase the good will always use it. Nowadays, a lot of nonprofit organizations and governments distribute the nets for free, which has led to an increase in coverage in Africa. Similar research was conducted on education programs that benefit up to five million students in India, and was quoted in the Nobel Awards press release. The Academy stated that the method of research now “entirely dominates development economics,” and “has great potential to further improve the lives of the worst-off people around the world.”
Amid the hype generated during such an important moment of celebration, concerns have also been raised regarding whether such methods are truly applicable in the field of economics. In the simplest terms, economic studies how society allocates scarce resources. This involves a broad and complex array of knowledge and topics. In contrast, RCTs focus on finding reliable answers by breaking questions down into manageable parts. The complex nature of humans and societies could mean that their interpretations (of RCT results) don’t hold very well in the real world. This is the thinking that underlies the controversy regarding external and internal validity in RCTs.
External validity refers to the universality of results. Can such research findings be effectively applied to broader contexts involving more societies and nations than were in the study? Probably not, as different societies are likely to respond differently according to their varying environments with distinct social and political situations (e.g. School based deworming will probably be a more effective education program in India, where soil-transmitted parasites are more prevalent, than in Libya). On the other hand, internal validity of such research (cause and effect) is also uncertain as societies and human behavior change over time, which means that the same results may not be replicated even on the same group of subjects previously studied. This raises concerns regarding how reliable such research and their corresponding interpretations are.
In addition to reliability, the methodology also poses questions about experimental methods. For example, RCTs require controllable factors in order to separate control and treatment groups. This means that RCTs can only be used to study private goods (i.e. must be purchased), which are exclusive in nature. Public goods would not be compatible with RCTs, because economists cannot prevent the control group from accessing it. Public goods include fresh air and knowledge, as a couple examples. This could be problematic when economists want to study issues such as air pollution. And even if economists can control which group of people enjoy the treatment, it would pose ethical questions. Different from RCTs in medicine, these social experiments do not involve consent from subjects. Performing experiments on the poor and deciding which population gets the benefits for the purpose of research would create ethical questions and social unrest. This was shown in Kerala, India, where local officials refused to use the trial to study about distribution of mosquito nets as the test discriminated against some families in the control group.
Practicing RCTs may not only be ethically problematic; it could also be financially burdening. Active intervention in societies can easily add up to large sums of money. A study showed that between 2012 and 2016, the average cost of RCTs in Switzerland was $72,000 USD. These experiments take up valuable public money and research resources that are not available in such impoverished regions. This means that such research is performed by the few who would be deciding which population or topic to study on, at a high opportunity cost (The resources could have been allocated to other experiments which have wider and more diverse impacts).
Nonetheless, every methodology has its shortcomings. Nobel Laureate Esther Duflo spoke in a Ted Talk on RCTs in 2010, in which she said:
“It’s not going to be easy. It’s a very slow process. You have to keep experimenting, and sometimes ideology has to be trumped by practicality. And sometimes what works somewhere doesn’t work elsewhere … There is no miracle cure, but modern medicine is saving millions of lives every year, and we can do the same thing.”
Poverty may not be improved as easily as taking a pill, and experiments on human societies are often complex and expensive. However, the successful examples of RCTs offer us some hope for the progress economists are making to create a better world for everyone to live in.
Featured Image Source: Business Insider
Disclaimer: The views published in this journal are those of the individual authors or speakers and do not necessarily reflect the position or policy of Berkeley Economic Review staff, the Undergraduate Economics Association, the UC Berkeley Economics Department and faculty, or the University of California, Berkeley in general.
