Payment by result: how to deliver result-based financing

What is payment by result (PBR)? An overview on its definition, key terms and conditions

Payments upon demonstrating a set of agreed results is an emerging contractual modality that many donors are choosing to pursue. DFID together with USAID are leading this discourse and applying this funding approach to an increased number of thematic areas, especially in health and education.

Knowledge of the context, risks and experience from past programmes are important conditions to propose a set of activities that can generate evidence of results on a continuous basis.
The condition for payment is in fact agreed upon a set of indicators that typically represent outputs and selected outcomes. At this time it is reasonable to assume that donors will increasingly focus on longer-term and structural changes, both in development and humanitarian sectors. For example, a choc of natural or political nature now require proposals that are multi-component/multi-area ranging from immediate response to primary needs all the way to livelihood rehabilitation linked to markets.

This is an interesting shift of focus from short-termism to more strategic programming that require rigorous evidence to be used in replicating or scaling up successful interventions.
For instance, a better service entails budgetary considerations and tracing the contribution in ensuring future funds becomes necessary. A possible way to achieve this is to communicate what works so to influence the debate of regulators about what works in target areas.
If similar objectives are now asked to be proven as condition to receive payments, development actors become accountable to show results that can be measured over time through multiple methodologies.

Reasons and challenges behind donor shift to a result-based approach to aid

The principle of putting evidence at the centre of payment flows is a bold and complex approach since some areas of change or attribution are hard to measure with a level of rigour that is satisfactory. For instance, 1) how to mitigate the risk of spill-over effects between two groups of recipients and non-recipients; 2) what is the percentage of recipients’ retention in an intervention and 3) what are the conditions associated to cash flows, risk-sharing and pre-financing requirements?
All relevant issues on the adequate method to measure behavioural or policy shifts should be discussed and debated in a payment by result modality from design.

Beside the existing bias of statistical validation to measure project achievements, there is an on-going opening from donors to negotiate on alternative ways to monitor and communicate what yield the greatest results. In some cases, the donor can agree to stop the requirement of tracking control groups (individuals receiving a different or no intervention) in view of the contextual barriers in following two distinct clusters of girls (Girls Education Challenge). In selected countries, the recipient group has been tracked for the whole duration of the project and evidence indicated a series of significant improvements in their condition. This example shows that an appreciation of the context allows for an open discussion with key donors on realistic targets and incentives to highlight observable change. Mastering this language opens a gateway to partnerships and it represents a vehicle for further funding.

It is likely that donors will continue to favour a greater commercialisation and result-based financing of suppliers; the inclusion of the private sector to further national is a clear indication of this trend.
At the same time, the payment based on evidence boosts the incentive to seriously consider management approaches focused on the use of evidence as a top priority.
Therefore, even if PBR has an inherent limitation in striking a balance between adaptive response and the ability to forecast and quantify results it also compels development partners to boost resources in systems to generate evidence real-time on what delivers value for money. Despite this, the effort to extract evidence should not overwhelm any partner but directly address the areas of measurable change that have been agreed and are under constant review with a key donor.

What are the implications for monitoring, evaluation, accountability and learning ?

In a PBR scenario, the business case to select the best indicators and trackable group of respondents is stronger than ever. An open discussion with donors and implementing partners on what is feasible and its costs requires a certain dose of conviction when advancing valid alternatives to experimental designs, which organisations can best substantiate given their long-standing presence in countries and thought leadership in certain methodologies. In support of this dialogue, there are also MEAL trends that confirm the need to trace contribution instead of measuring attribution when results ought to be reported, particularly if related to social transformations or modifications of regulatory frameworks.

The agreement with donors on how to generate and use evidence to inform activities can lead to a renewed management approach. For instance, if a particular training showed more effectiveness once repeated, the project lead should discuss with the donor on how to reallocate resources to reinforce a specific information or investment on set of skills that proved to cause remarkable improvements. A constant review of evidence and assumptions might even sustain some degree of standardisation in tackling particular conditions or feedbacks (Core Humanitarian Standards).
In other words, the fluidity of many contexts implies that a set of assumptions and risks (internal and external) need to be well-appraised at design and constantly tracked along the whole cycle of the intervention. If a new government comes to power, will it remain accountable to policy reforms agreed previously thanks to the engagement of donors and their partners? If a pre-condition is favourable at design, how would a sudden change in power dynamics either boost or restrict the scale of interventions in domains like social activism or minorities’ protection?

Interventions shall then embed triggers to adapt to possible contextual variations and a superior level of rigour when revealing their implications to target population. Multiple courses of actions need to be identified and considered at inception to outline scenarios in case these shifts occur.  Thus, MEAL vests a critical role in adjusting project objectives based on the evolving circumstances. Evidence of changes and the perceived quality/relevancy of activities by beneficiaries require a link to contextual fluctuations, particularly if of political nature.
The public from donor countries expects that sustainability of development interventions is borne by recipients’ capacity to resource the continuation of funded initiatives, preferably via market-based solution. To deal with this demand and result-based prerequisites, solid and well-resourced MEAL frameworks can validate in real-time if a sequence of activities is triggering a structural change and the roles of multiple stakeholders in that process. With it, the ability of programme managers to collaborate closely with MEAL advisors signifies the foundation for learning when extracting, synthetizing and sharing data from the field.

Areas to invest on to establish performance benchmarks and result-based delivery

The ability of organisations to manage a fund or to deliver technical assistance within the context of PBR-like arrangements is emerging due to contracts established on output and outcome targets in the domain of girls’ education and resilience building.

The key steps to consider are an overall improvement in processes to handle similar contractual arrangements and a way to track systematically all project experiences that contain examples of adaptive response and strong MEAL frameworks. Their compilation can support a management toolkit and skills growth strategies to enable teams involved in programmes to: 1) articulate indicators clearly, 2) link inputs to long-term changes, 3) resource MEAL sufficiently via large monitoring teams, 4) focus on updating context-based assumptions and the theory of change.
All these actions are the way forward to respond to PBR requirements if coupled with a progressively deeper understanding of an area of intervention from previous programmes that have been implemented by development organisations. What donors seem inclined to favour in PBR is the identification of what has worked in the past within a defined geography and its replication or scale-up based on a market opportunities and local circumstances/risks affecting target groups.

Learning from past projects on how management choices delivered the most cost-effective activities linked to a theory of change can position any organisation as supplier.
In international development, it is reasonable to expect that plenty of examples can inform the debate on “measurability” but most importantly they can consolidate recognition of how past achievements were attained or exceeded.
Some organisations hold impressive portfolio remains and their work is to distil a system that can respond and even influence the very definition of payment by result and outcome areas.
The wealth of evidence that is within reach is surely the next big opportunity to engage donors in a deeper understanding of where flexible programming and testable assumptions meets conditions for payment and validation of a programme theory.

Advertisements

Adaptive management to achieve impact results

One of the trendiest buzzword in the development and humanitarian sector at the moment is “adaptive management”, which carries heavy-weight in focusing on MEAL practices while remaining neutral to political forces and the increased commercial pressures upon aid spending.
But what does adaptive management mean in practice and what are the key considerations that we should appreciate when we come across this notion?
To respond to these questions, the next few sections are meant to uncover key concepts associated to adaptive management and a concluding remark on how they could be understood in the present institutional donors’ landscape.

1: Recognising the importance to use evidence when taking management decisions

One of the greatest benefits from institutional donors that are increasingly focusing on adaptive management is a widening recognition of the need to learn, iterate and adapt.
Particularly when a programme presents multiple components or when the context is volatile, project managers are supposed to enable rapid changes as soon as the evidence points out at sub-optimal processes, lower results or changing forces in the context of intervention. These types of project adaptations already happen and they are based on numerous factors.
The way to act based on evidence varies significantly depending on willingness and expertise of programme managers and MEAL focal points to work together and review multiple kind of information (financial, programmatic, operational…) in several occasions during a project life cycle. The current trend indicates that input financing is not enough, long-term changes results need proof.

2: Design a management approach that is established on adaptive practices

A recent study commissioned by BOND indicated there are two dimensions to consider when choosing between a more rigid or flexible programme approach. In fact, there are two kind of information needed at the design of a new project: knowledge of what causes change and knowledge of the context where to implement activities. By addressing these two important questions, it will be easier to understand the kind of data to inform how decisions are taken.

For example: do we know enough about the target groups and their needs? Or, do we have enough equipment to improve a certain health/food issue in the areas with the most acute exposure to it? Are we addressing a climate shock or a long-term drought, do climate extremes alternate?

ttt

As the table above shows, the more we know about the context and what causes results, the more linear the implementation will be. However, in many cases the issue the project seeks to address is caused by unpredictable shocks that require fast response or complex development issues that require a lot of learning to address effectively. Selected health outcomes are instead more measurable as scientific methods are applicable to the treatment that is typically fully controllable (e.g. vaccines). All these variations inform the management style and the speed needed to make decisions either to adapt to evolving risks or to consolidate results. In both scenarios, data and adequate methods to inform the programme manager are necessary.

3: Design is the beginning of success when it sets realistic outcomes and targets

For the reasons outlined above, the choice of suitable and feasible outcomes is necessary at design.
Since outcomes and impacts represent long-term changes: empowering women financially and socially, building resilience to climate change, reducing chronic food insecurity etc., the ability to forecast and attribute these kind of behavioural or social transformations can be intricate without iterative learning about the context and solutions to the issue under consideration.
Linear expectations can be threats in particular environment and a reduction of social risks among target communities when adopting new behaviours are mitigated in a successful programme.

4: Adapt to donors by expanding monitoring from short to long-term changes

Deciding how transformational an intervention could be to address SDGs also requires a full understanding of the direction of travel set by influential donors in delivering aid.

In the table below, DFID graph explains a shift from 100% funding upfront to 100% funding on delivery. This change has profound implications on the modality to use aid since constant evidence review – at times hard to access- will be critical to adapt management approaches in implementing project activities that results to be more transformational than before and with larger outreach.
Managers are supposed to start considering rigorous use of mixed methods (qualitative and quantitative evidence) to inform quick responses to changes in contextual forces or assumptions on how long-term changes are expected.

gg

Traditional ways to measure change through evaluation surveys and loose monitoring systems are unfit when demonstrating long-term changes. A culture of proof need significant organisational shifts to prioritise skills growth in generating and analysing evidence of structural changes and their scale in intervention areas. A possible approach could be strategy testing, focused on monitoring the interests and incentives driving key actors and linking them to efforts to promote critical reforms. Because bringing outcome changes equalise to navigating an unpredictable and complex landscape of interests, plan of activities cannot be rigid and need constant feedbacks and reviews.

5: Final remark: Develop negotiation strategies with stakeholders and funders

One of the first questions that could come up is: why all these complexities are embedded in contracts when there are clear needs in a context of new or chronic shocks?
The reason is that a transition from reactive to forward-looking aid entails greater risks. It is hard to predict and forecast in numbers long-term changes while political pressures and public ownership of development solutions are requesting greater scale of results in depth and breadth.
There is greater agreement around situations of immediate response, to a natural shock for example, but it branches out in many directions when a solutions need to apply to chronic marginalisation and less tangible needs, like empowerment and legal rights.
In figuring out what works, there is also scope and space to negotiate since donors themselves recognise the need for constant learning in volatile and unpredictable contexts. They are open to resource ways to access adequate evidence and inform all their stakeholders.

NGOs along with other development partners remain responsible and in a legitimate position to request adequate funding that are linked with credible monitoring, evaluation, accountability and learning systems. How to frame and cost for them from operational conundrums need to become a strategic choice of innovative ways to boost dissemination of complex data in simple and compelling form.
The negotiation will be finally successful when evidence from programmes will be used across multiple teams so that, the story of change will be a narrative with multiple facets but based on the same well-appraised facts from the field, where change happens.