A CSO critique of the preliminary findings of the World Bank 2012 Development Policy Retrospective calls for policy changes to strengthen environmental risk assessment, improve accountability for results and enhance governance.
As the World Bank gears up for the negotiations for the 17th replenishment of the International Development Association fund (IDA-17), the Bank’s Board development committee (CODE) will be discussing one of its main contributions – Development Policy Operations. CODE reviews the results of a three year retrospective of Bank’s Development Policy Lending Operations, which since 2004 have represented close to a third of the Bank’s overall lending (and 20% of all IDA lending on average). The 2012 DPO Retrospective focuses on risk assessment and management, results and operational policy 8.60 reforms. the Retrospective makes a number of interesting and useful arguments about trends in the use of DPOs. presents an important opportunity to reflect on the role and effectiveness of DPOs as a development instrument in the context of the IDA-17 negotiations and the preparation of the IDA-16 mid-term report. To that end, the Retrospective has targeted some of the key issues.
We welcome the focus of the 2012 DPO Retrospective, particularly the emphasis on risk assessment and management, results and OP reforms. Based on the presentation of the concept note and the preliminary findings, the Retrospective makes a number of interesting and useful arguments about trends in the use of DPOs. The report presents an important opportunity to reflect on the role and effectiveness of DPOs as a development instrument in the context of the IDA-17 negotiations and the preparation of the IDA-16 mid-term report. To that end, the Retrospective has targeted some of the key issues.
In comments submitted by Bank Information Center, in collaboration with Fundar, Center for Analysis and Research A. C. (Mexico) and Derecho Ambiente y Recursos Naturales DAR (Perú), based our analysis of several environmental DPOs in Latin America and the presentation of the concept note and preliminary findings, we underscore the challenges to maximizing the effective use of DPOs, consistent with the Bank’s commitment’s to Paris Declaration and Accra action plan on aid effectiveness. Strong and accountable environment and social risk assessment and management for DPOs remains a concern. It is important that the Bank seems to have acknowledged that certain DPOs pose higher risks than others. However, the described review process for environmental risk leaves substantial concerns that more predictable, transparent and accountable risk assessment requirements and procedures for DPOs are needed. We know less about the results of DPOs from the Retrospective than we should. The preliminary findings appear to elevate this concern about the quality of evidence of DPO impact, but does not address it. Monitoring and evaluation of DPO results should become both a more visible focus of DPO design and implementation. Transparency, participation and accountability represent areas where some progress has been made with DPOs, but much more is needed. Finally, these challenges will require more than incremental modifications to DPO operational policy 8.60. A wholesale review of OP 8.60 is necessary with full consultation provisions to update and align the requirements and procedures for DPOs with other policy changes underway at the World Bank.
To download the full comment, see:
CSO Comment on World Bank 2012 Development Policy Operation Retrospective (BIC, DAR, FUNDAR), Sept 1, 2012Unfortunately, the Retrospective does not appear to delve more deeply into these trends or provide much analysis of actual DPO performance in the areas of risk management or results. Much of the information on DPO trends is carried over from past Retrospectives (gross indicators of approved operations, decline in the total number of prior actions, correlation between IBRD borrowers with stronger CPIAs and shares of DPO lending). Various opportunities to explore further below the surface of these trends do not yet appear to have been prioritized.
Several examples of where the report might provide more descriptive or analytical detail regarding DPO use include:
- Regional or sectoral forecast of likely DPO use in the next five years, with sub-regional governments, for example
- Factors explaining the decline in DPO use in IDA countries, particularly during the financial crisis and in light of the upcoming IDA
- Possibilities for application of Operational Risk Assessment Framework (ORAF) to DPOs in the future to address risk assessment challenges, particularly for contexts of weak governance
- Variation in the combined use of DPOs, Technical Assistance Investment Lending and other Bank instruments on DPO performance
- Deeper assessment of DPO use in new sectors (environment, forestry)
- More on results and methods for strengthening accountability for DPO results
- Actual evidence of improvements in transparency and consultation associated with DPOs (such as instances of simultaneous disclosure in 2011-2012)
- Factors explaining consistent overrating of DPO performance compared to limited IEG reviews
- Transaction and development costs and benefits of DPOs (average preparation and supervision costs, other costs and benefits related to partnerships)
As indicated in several meetings with OPCS and in submissions of analysis of past DPOs, we strongly believe that the preliminary results of this Retrospective tend to be descriptive rather than explanatory in nature, suggesting the value of an evaluation of DPO performance. That said, we don’t believe all of our concerns can be addressed by a Retrospective report.
The comment highights the following issues:
1. Risk Assessment: The current operational policy OP 8.60 does not provide transparent or verifiable criteria for the identification and classification of environmental and social risks and potential impacts of DPOs. We believe that this leads to underestimation of these risks and impacts. In the update of the Bank’s safeguard policy framework, the Bank should ensure consistent and effective classification of environmental and social risk, including DPOs, under the criteria currently in OP 4.01 or some equivalent single policy that covers all Bank instruments. As with the PSIA, the Bank’s environmental assessment policy should further specify requirements for the use of strategic environmental and social assessment (SESA) for prior actions that are likely to have significant effects. As the Bank contemplates expanding the use of an integrated risk assessment framework (ORAF) for DPOs, stakeholder consultation regarding the design and before piloting the use of such a framework is recommended.
2. Results: The Retrospective should clarify what is known about DPO results, indicating gaps where they exist and strategies to fill these gaps that range from methods for impact evaluation to greater targeting of borrower M&E systems as prior actions in DPOs. OP 8.60 should provide a more robust commitment to a transparent and meaningful public discussion of the ambition and relevance of the chosen outcome indicators and oversight mechanisms to monitor and support implementation. DPOs should provide a better assessment of the costs and benefits of DPOs is required. The Bank should clarify how DPO impact assessment will be conducted in the future.
3. Reforms to OP 8.60: As indicated above, wholesale rather than piecemeal reforms to the requirements and procedures for consultation and participation, social and environmental risk assessment of DPOs are needed and for which few stakeholders are made fully aware in advance. Ideally, a full review of OP 8.60 would be properly aligned with the two year Safeguard Policy review. The Bank should proceed in the direction of strengthening the standard on budget transparency in broad consultation with stakeholders, including CSOs.
4. Governance of DPOs: Governance remains a concern that the Retrospective should address in greater depth than is indicated by the preliminary findings. Improved consultation and transparency is necessary to ensure legitimacy and likely achievement of chosen DPO prior actions and triggers. DPOs are not adequately transparent or properly consulted, which in turn undermines the quality of the design and outcomes. Strengthened policy requirements and improved guidance on minimal standards for acceptable transparency and consultation is required that establishes greater advisory capacity by a wider range of stakeholders at key moments in the project/reform cycle but provides flexibility for Bank-client negotiation. These standards should clarify reporting criteria for DPOs that include mandatory translation of program documents, disclosure of the policy matrix or results framework prior to Board approval, qualitative evaluation techniques, wider use of stakeholder feedback mechanisms, consideration of how external accountability mechanisms could become more routine with DPOs.
5. DPO Evaluation: The Retrospective is simply not designed to answer some of the questions posed in these comments. The effectiveness of DPOs can really only be assessed through evaluation methodologies and the independence of the investigator. A full evaluation of DPO performance is long overdue, given the Bank’s dependence on this instrument and the eventual reforms to some aspects of OP 8.60.