Based on a literature review on Monitoring and Evaluation (M&E) approaches (presented in Deliverable 1), a Performance Measurement System for energy cooperation project (PMS) model has been developed. The tool is based on the most known and shared evaluation criteria, even if new theoretical approaches have been introduced to allow a more complete quantitative evaluation of energy cooperation projects. Based on the DAC-OECD1 Results Chain (Inputs – Activities – Outputs – Outcomes – Impact).
PMS is composed by two steps:
- Internal Evaluation – result based/project oriented, that assesses project’s direct performances and is focused on the project’s results and consumption of resources. It allows the evaluation of the first four DAC-OECD Criteria (i.e. Relevance, Effectiveness, Efficiency and Sustainability), which may all be assessed within project boundaries and timeframe;
- External Evaluation – development based/people oriented, that analyses the long-term Impact of the project on the target community’s five Capitals, i.e. Natural, Physical, Human, Social and Financial (evaluating the last DAC-OECD criterion: Impact).

Deliverable 2 presents the theoretical approaches, methodologies and tools adopted for each step of the model. The three following deliverables validate and enhance the PMS method through several case studies. Deliverable 3 analyses a HydroBioPower project, implemented in Ethiopia by an Italian NGO, and a hydroelectric project, funded by the Italian Ministry of Foreign Affairs in Ethiopia. The implementation of the model to these cases highlighted the versatility of the model that has been applied also to projects evaluated earlier, thus demonstrating the consistency of results through a more quantitative and graphical representation. Deliverable 4 analyses a project just launched by Enel Green Power in Ollagüe (a village located in Chilean Atacama desert where an hybrid mini-grid has been installed) and a project implemented in Malawi by an Italian NGO with the objective of disseminating improved cooking-stoves. In the first case (Ollague) the PMS model has been used only to forecast the expected impact. The second project (Malawi) has been analysed under both points of view (internal and external evaluation). In light of the results obtained, Deliverable 5 integrates the model with two key improvements, that make it more effective and its results more suitable to be analysed and compared:
- Revised measurement method of the project performances (Relevance, Effectiveness, Efficiency and Sustainability) allows the evaluator to better contextualize the obtained results, comparing these criteria with internationally recognized benchmarks or national standards. The aim is to use them for a comprehensive analysis of the project with respect to current energy policies or expected objectives and, in a broader view, to better guide future project design;
- Inclusion of a mandatory set of indicators (Alerts) within the Impact evaluation: this improvement allows the evaluator to take into account all fundamental issues that characterize every energy project. Alerts represent the minimum indicators basket that should be always monitored and evaluated, also when no project-dependent indicators are specified.

The improved version of PMS model has been used to analyse two additional projects: “Powering Education” and “Barefoot College”: the first is developed by Givewatts and Enel Foundation in Kenya and the latter is implemented by Enel Green Power in collaboration with Barefoot College (India).

The document ends with a revision of results, applying the use of the Alerts on the analysis conducted on “Ollagüe” project: this revised analysis demonstrated that the use of mandatory Alerts did not influence the results in their main features, but allowed to have a more structured and complete evaluation of impact.

Link

Source: oxygen Oxygen magazine article