This post originally appeared on the USAID Learning Lab blog.

by Lori Ashford with contribution from Soumya Alva and Reshma Naik

Most global health and development projects include a capacity-strengthening component, but how well do they measure whether greater capacity was achieved and the underlying determinants of performance? Capacity development activities (the inputs) should ultimately lead to improved performance (the outcome). But what are the best tools and indicators to measure the outputs—the additional capacity—that will lead to the performance outcome?

To answer this question, a landscape review was conducted by MOMENTUM Knowledge Accelerator, one of six global awards under the USAID-funded MOMENTUM umbrella, which works with countries to accelerate improvements in maternal, newborn, and child health services.

Shortfalls of Existing Tools

The review found that most existing tools fall short when it comes to measuring the many dimensions of capacity that are critical for attaining an entity’s end-goals. For example, the tools:

• Tend to focus on organizations and exclude other units of analysis, such as systems, networks, and communities that influence health outcomes.
• Often rely on self-assessment and make little use of evidence.
• Focus on latent (potential) capacity rather than on performance and yield little information about the underlying determinants of performance.
• Are often not linked to an explicit theory of change.
• Are often ill-suited for highly complex environments in which systems are decentralized, tasks are not straightforward, and interactions are dynamic and may vary depending on the situation.

Understanding “Capacity” and How it Links to Performance

To measure capacity effectively, we need to understand its attributes. At its core, capacity involves the ability to do and achieve things and make changes as needed, and capacity development entails identifying and addressing areas of performance that need improvement. An important dimension of capacity development is improving resilience. Why resilience? Because it allows individuals, organizations, and systems to thrive in complex and unpredictable environments—that is, they can anticipate risks, cope with adversity, and recover in a timely and effective way.

Capacity can be understood as the attributes that enable an actor working at any level (system, organization, community, or individual) to perform well in conditions of high complexity. Ideally, capacity development also builds “social capital”—that is, social cohesion, trust, and a sense of shared mission—because it is a source of vitality, growth, and effectiveness for entities that promote public health.

Choosing the Right Capacity Measures

Appropriate measures that capture the role of resilience, performance improvement, and capacity at various levels (individual, community, organization, and system) are needed to examine the effect of capacity strengthening for organizations working in complex environments. These measures must be reliable, easy to use, and able to produce actionable results. They need to focus on the effects of interventions and program activities on outcomes, performance, and new solutions using low-touch complexity-aware monitoring methods to collect and analyze information. The four types of measures involve compiling evidence from:

• Organization level entities such as local partner organizations that modify programs to better reflect locally prevailing social norms, values, beliefs, and practices that influence health outcomes.
• System level entities such as health care facilities or local NGOs that engage in health care promotion or service delivery that modify programs to better reflect locally prevailing social norms, values, beliefs, and practices that influence health outcomes.
• Health care workers who use client or other feedback to improve program reach, coverage, or effectiveness.
• Community-based service provision facilities that use client or other feedback to improve program reach, coverage, or effectiveness.

Data collection approaches for these measures can be adapted based on the nature of project activities and systems, as well as available resources. In some cases, it may involve questionnaires and responses while, in others, a simple review of written descriptions. Importantly, the proposed indicators focus on practices that, when performed regularly, are directly linked to improvements in program quality, coverage, and effectiveness.

Lessons about Assessing Capacity

The review also identified important lessons about capacity assessment, gleaned from a study that examined the experiences of 20 NGOs that applied one capacity-assessment tool, the Discussion-Oriented Self-Assessment (DOSA), for four years.

The assessment tool or methodology should:

1. Be integrated into a broader process of transformation that involves setting priorities for capacity development, planning the activities, and measuring results.
2. Be grounded in a theory of change that is consistent with the entity’s mission and objectives.
3. Shed light on the causes of under-performance, which could include inadequate knowledge or skills as well as low motivation, a lack of incentives, weak supervisory support, insufficient resources or other factors.
4. Help users identify how various capacities (for example, service delivery, external relations, and strategic management) relate to each other to enhance overall performance.
5. Promote rich dialogue among participants. This greatly enhances the assessment process because the insights gained can foster learning and create a shared vision to guide future decisions. Dialogue builds social capital, which in turn enhances performance in mission-driven entities.

Well-designed capacity assessment tools are themselves part of capacity development; the tools should help participants think critically about the what as well as the why of performance. The assessment process should help users create an enabling environment for consistent and ongoing demonstrations of capacity. An important outcome of capacity assessment is learning, which is a cornerstone of resilience and the ability to adapt.

And, speaking of adapting, every project must develop its own assessment approaches and indicators that are specific and tailored to its overall objectives. If program staff developing these tools were to incorporate many or all of the lessons from this landscape review, capacity-development initiatives would become more effective.

This blog is based on the work of Beryl Levinger. To learn more, read the full MOMENTUM paper titled Measuring and Assessing Capacity: A Landscape Review (2021).