SOCIETÀ ITALIANA DI DIRITTO ED ECONOMIA
Pietro Ghirlanda (University of Pavia)
Enrica Chiappero-Martinetti (University of Pavia)
Abstract
Research Context
Disadvantaged and fragile countries are historically threatened by a series of multifaceted and interlinked crises which create great stress on individuals’ and communities’ capabilities: extreme poverty, humanitarian challenges, sharp inequalities, pandemics, violent conflicts, climate change, etc. Among these crises, one that is often forgotten is a measurement crisis concerning the assessment of development projects. Project assessment is in fact crucial for understanding the effectiveness of actions and commitments taken for addressing these kinds of emergencies, deciding how and where to allocate the available resources, and designing future interventions aimed at improving the multiple dimensions of human wellbeing. However, it is deeply influenced by the institutional context that establishes the “rules of the game” for development interventions.
In this sense, the main problem is that the methodology which currently represents the “gold standard” in development economics and informs development policies, i.e., randomized controlled trials (RCTs), is difficult to apply in some important circumstances: a) when development programs are broad in scope and aim to ingrate economic value creation with social and environmental sustainability, b) when they have been already implemented and thus it is required a mostly retrospective assessment, c) when their effects unfold in a medium/long-term horizon and are not limited to the short-term, d) when their implementation in a certain domain is likely to produce cross-sectorial and cross-thematic spill-overs in other domains, e) when an operational approach to evaluation, suitable for capturing governance issues and the degree of participation of target groups in projects’ implementation, is equally relevant.
Accordingly, the partial merit of RCTs of being a consolidated, scientifically rigorous, and easily replicable method goes together with some limitations that risk compromising the same aim of development policies and creating a sort of “technocratic illusion.” Namely, being characterized by the necessity to create a manageable experimental setting to isolate a few measurable variables and assess the impact of a “treatment” on a restricted sample of participants vis-à-vis a control group that does not receive the treatment, it can be tricky for RCTs to achieve external validity (Achen, 2022). Moreover, other elements that bring their reliability into question are their feigning ignorance about the content of experience, common knowledge, and best practices (Pritchett, 2020), their lack of a holistic perspective, and their being guided by a rule-utilitarian inspiration indifferent to distributive concerns (Henderson, 2022). In particular, the so-called “equipoise” requirement, i.e. conducting experiments only when there is reasonable uncertainty over the benefits of a treatment—without creating disadvantages in the control group—is difficult to satisfy in development contexts.
In light of this scenario, this paper aims to propose the capability approach (CA) as a richer informational basis for assessing development projects and going beyond reductionist approaches. Moreover, by capturing important aspects often ignored by traditional evaluation instruments, such an approach could also help to reform the institutional context of development policies and consequently orient funding allocation and project design more effectively.
Methodology
The CA is a social justice theory that focuses on the evaluation of human development in terms of the substantive opportunities people have to achieve doings and beings they have reason to value in several dimensions of their lives. Hence, its constitutive multidimensional and complex character, which allows researchers to properly weigh the heterogeneity of both personal and social conversion factors as well as of the role of individual agency and empowerment, and its vagueness, which transforms the evaluation of achievements from a binary issue into a matter of degree, make it particularly suitable for the assessment of contested subjects like development interventions (Clark, Biggeri, & Frediani, 2019; Garcés-Velástegui, 2022; Pham, 2018).
Accordingly, this approach offers several advantages: a) it provides a unifying lens through which to assess different case studies and compare projects’ real effectiveness while highlighting their specificities; b) it enables making more detailed analyses and giving priority to the quality of life of the worst-off by placing individuals and communities into their respective contexts, instead of focusing just on the maximization of average utility; c) it encompasses not only direct, short-term, tangible benefits but also more long-term, indirect, intangible ones; d) it consider the process through which outcomes are achieved rather than only outcomes themselves.
Considering these advantages, our paper takes inspiration from the CA and proposes a multidimensional framework aimed at rethinking how development interventions are assessed and the central elements around which development policies are developed. The framework in question consists of six dimensions:
1) the ownership of development projects and the partnerships that can guarantee their success;
2) the participation and level of engagement of local actors, either formal or informal, as well as the governance mechanisms implemented;
3) to what extent individual and community agency and empowerment have been activated by the projects;
4) what have been the tangible and intangible impacts generated in the short, medium, and long run;
5) what have been, if any, the cross-sectorial and cross-thematic spillover effects produced;
6) the environmental, social, and economic sustainability of the projects.
Moreover, the paper tests this framework by applying it to a specific case study: the assessment of three interventions that have been run in Kenya by AICS (the Italian Agency for Development Cooperation) concerning the important issues of maternal and child health and tuberculosis infection. The projects assessed are the SonoMobile, the HIPS-TB, and the ATI-TB projects. Each project is analyzed through a mixed methodology informed by the research framework, combining retrospective quantitative surveys and qualitative in-depth interviews conducted with a purposive-selected sample of the main projects’ beneficiaries. In this sense, the analysis of our case study confirms how applying the capability approach offers the opportunity to develop not a deterministic method but a holistic way of thinking that can be adapted to different contexts and operationalized through various methodologies in compliance with the initial objectives of different development interventions. In this sense, it can help to better understand the strengths and weaknesses of the projects under analysis and contribute to orienting future funding allocations and project design.