Results-focused Project Design and Management » M&E

M&E: What Else Can we Improve?

admin — Wed, 11 Apr 2012 01:00:31 +0000

You may have either been involved, practicing or victims of M&E. Some for more than a decade or two. The practice dates back to the Marshall Plan and borrowed from the Military establishment. Over the years, it evolved, albeit incrementally to what is practiced and understood by you and me today. Its focus expanded and changed depending on who needs (mostly the funders and less so, the beneficiaries) the results. Initially, it focused on benefits, then impacts, then results (today), then learning and who knows what the next would be. Like me, many of you are asking: So what else do we not know about M&E?

This article is by no means a comprehensive. There are lots of theoretical, manuals, toolkits, and guides written on the subject. Most practitioners (and users) are still looking for something missing. The current practice is still too weak, unreliable and the results do not effectively feed into the original design to enable real-time adjustments and shifts in implementation strategies.

In her most recent TED presentation, Ms Melinda Gates observed that lots can be learned from the private sector that can improve M&E and project management.

Here is our list of four.

1. Feedback to and from beneficiaries must be shortened. Today, M&E results take an awfully long time to be collected, reviewed by the stakeholders and delivered (if at all!) to the beneficiaries so they can act on the freshly collected data on how everyone in the project is doing (and learning!). The longer the data reaches the beneficiaries, the more useless they become and the less impact they will have on the changes and transformation expected.

2. Beneficiaries (and not the project implementers) should review and decide what to do with the M&E data. The intention is always to make the beneficiaries use the information and eventually do what needs to be done.

3. Usually, the delivery channel in planning the projects (check the latest DMF you facilitated if in doubt) is assumed to exist (or need not be improved at all). The project proponent must have such a channel. Omitting the channel does two things: we forget that they facilitate the delivery and collects data on the results and utility of the project outputs. The more real-time feedback is collected, the faster the project can adjust its approach to the contexts and real needs of the beneficiaries.

4. Existing ICT technologies are taken for granted. ICT can enable the implementers to access real time data and feedback from the beneficiaries. We now have so much choices as in the 1900’s.

Can you still add more to the list? Will a new paradigm for a more real time M&E be developed? Tell us what do you think?

Stay with us! We will share more M&E techniques in the forthcoming editions.

Evaluation for Equitable Development Results: Out Now!

admin — Wed, 28 Mar 2012 01:30:55 +0000

UNICEF, in partnership with UNDP, UNWomen, ILO, IDRC, IOCE, IDEAS and CONEVAL, are pleased to inform you that the book “Evaluation for Equitable Development Results” is now available for free download here.

This new book complements the manual “How to Design and Implement Equity-focused Evaluations” available here.

“Evaluation for Equitable Development Results” aims to contribute to the international debate on how the evaluation function can contribute to achieving equitable development results by conceptualizing, designing, implementing and using evaluations focused on human rights and equity. It does so by offering a number of strong contributions from 27 worlds-level experts and senior officers in institutions and governments dealing with development and evaluation. The authors also presented their contribution in a series of webinar. You can download the podcasts here.

Please find below the table of content:

Prefaces:

Colin Kirk, UNICEF, Director, Evaluation Office
Soma de Silva and Jim Rugh, President and Vice President, IOCE
Ray Rist, President, IDEAS

Part 1: Evaluation and equity

Evaluation to accelerate progress towards equity, social justice and human rights.
by Marco Segone, UNICEF Evaluation Office; Co-chair, UNEG Taskforce on National Evaluation Capacities; and former IOCE Vice President
Human rights and gender equality in evaluation.
by Janie Eriksen, UNICEF Evaluation Office, Shravanti Reddy, UNWomen Evaluation Office, and Janice Muir, OIOS; and members of UNEG Taskforce on Human Rights and Gender Equality
When human rights is the starting point for evaluation.
by Donna Mertens, Department of Educational Foundations and Research, Gallaudet University
Strengthening Equity- focused evaluations through insights from feminist theory and approaches.
by Katherine Hay, Senior Specialist, Evaluation, International Development Research Centre, Regional Office for South Asia and China
Decolonizing evaluation in a developing world. Implications and cautions for Equity-focused evaluations
by Rodney K. Hopson, Duquesne University, Karen E. Kirkhart, Syracuse University and Katrina L. Bledsoe, Education Development Center, Inc

Part 2: Methodological implications for Equity-focused evaluations

Methodological issues to design and implement equity-focused evaluations.
by Michael Bamberger, Independent consultant
Developmental evaluation for Equity-focused evaluations.
by Michael Quinn Patton, Founder and Director, Utilization-focused evaluation
Systems thinking and Equity-focused evaluations.
by Martin Reynolds, independent consultant and Bob Williams, independent consultant
Methodological challenges in using programme theory to evaluate pro-poor and equity-focused programmes
by Patricia Rogers, Royal Melbourne Institute of Technology University and Richard Hummelbrunner, Independent consultant
Case Study and equity in Evaluation.
by Saville Kushner, University of the West of England
Values-Engaged Evaluations.
by Jennifer Greene, University of Illinois at Urbana-Champaign

Part 3: Examples of Equity-focused evaluations

Evaluating the contribution of UNDP to equity-focused public policies in Brazil and China.
by Juha Uitto, Deputy Director, UNDP Evaluation Office and Oscar Garcia, Senior Evaluation Adviser, UNDP Evaluation Office
Using a human rights approach to evaluate ILO’s discrimination strategy.
by Francisco Guzman, Senior Evaluation Specialist, ILO Evaluation Office
CONEVAL experience in evaluating interventions for Indigenous populations in Mexico.
by Gonzalo Hernández, Thania de la Garza, María Fernanda Paredes and Brenda Valdez Meneses, Consejo Nacional de Evaluación de la Política de Desarrollo Social
UNICEF supported evaluations with elements of equity-focused evaluations.

M&E Systems & Your Cellphones

admin — Thu, 19 Jan 2012 08:20:35 +0000

You guessed it! Project management is discovering more and more uses of the power of the ever present cellphones. Like the internet, cellphones reached the remote rural areas much faster than refrigerators, TV, radio and even electricity did.

Now monitoring and evaluation (M&E) has invaded the cellphone space and is now gaining traction among development facilitators. They are joining the bandwagon for cost effectiveness and the fun of using a technology for the usually tiring and boring ME chores. In this method, the beneficiary-respondents use GMS-enabled phones as point of interest (POI) input devices that feed to a central data server via a database template for aggregation, visualization and even re-dissemination of the processed data over the internet. PMUs can do the four simple steps: (a) create an account, (b) download and install a software, (c) collect location data and (d) upload, and view and analyze over the web portal. The process is simple, paperless, and guarantees real time data monitoring. It allows for instantaneous response and actions from the field.

POIMapper® is one such system. PoiMapper is a mobile POI data collection and sharing solution. It is suitable for development projects focusing on: health, education, agriculture, forestry, mining, governance, land use management, climate change, disaster management and for commercial applications like installation and maintenance of telecommunications equipment. It is offered with a complete cost-effective SaaS (software-as-service) model and is provided by Pajat Management and global partners, who also provide related services such as training, on-site support and organization/sector-specific customization.

With Plan International, POIMapper has three installations: Thailand, India and Kenya. You can download a full description in this site.

For a limited time, individuals and organizations can avail of a free POIMapper installation starting here.

Participatory Planning, OK….What About Participatory M & E?

admin — Fri, 17 Apr 2009 14:53:28 +0000

Much has been said about participatory project design and planning and its importance in achieving project results. However, Results-based Management is also about monitoring targets in the midterm. And if we want an accurate picture of how our project is really performing, we should get the involvement of key project stakeholders during the exercise. This is the essence of Results-based Participatory Monitoring and Evaluation (PM&E) according to Francoise Coupal. He further explains that PM&E should be results-based in that close attention is paid to monitoring and evaluating the achievement of results vs activities. And adjusting project strategies and activities to better meet project results is one of the critical outcomes of PM&E. For more on Mr. Coupal’s article click here.

Other readings on PM&E, as well as some case studies and field experiences are available in these links:

http://www.ibsea.org/allincbnrm/components/com_remository_files/downloads/Tracking%20Change%20Together.pdf

http://intercooperation.org.in/images/icindia/Part.pdf

http://www.coady.stfx.ca/resources/abcd/SEWA%20PME%20Manual.pdf

And for those who are interested to undergo formal training and who don’t mind travelling to the Philippines, a 3-week course is being offered on PM&E. See link.

Training of Facilitators in Hue, Vietnam Completed

admin — Mon, 06 Apr 2009 14:50:35 +0000

Another Training of Facilitators in Results-focused Project Design and Management has been conducted at the Pilgrimage Village Hotel in Hue, Vietnam. The event ran from 12-20 May and was attended by a good mix of international participants from different backgrounds. The group, composed of individuals from Afghanistan, Bangladesh, Laos, Sri Lanka, Philippines, Singapore, and a host of project managers from the Vietnam government, has finally completed the nine-day training workshop which included sessions on the project design tools and facilitation inputs.

Three Softwares for your Job

admin — Mon, 30 Mar 2009 08:09:02 +0000

In this article, we like to present three Results-Based Monitoring and Evaluation Software that you can use in facilitating or supporting PPMS/DMF initiatives in your respective work areas. These are: (a) LogFrame for Windows 1.0, (b) AidProject M+E and (c) (PACPLAN) Solution.

Logframe for Windows 1.0 was developed by the Maizemoor International, whose founder has established some of the most successful and sustainable results based management (RBM) systems in international technical cooperation. Specifically, the company designed and installed the RBM systems for the German Technical Cooperation Agency (GTZ) and the British Department for International Development (DfID). It is a simple and flexible way to put the Design and Monitoring Framework (DMF) Approach in the heart of your project design, monitoring and evaluation.

Recently, the LogFrame for Windows 1.0 has been streamlined to focus on just the DMF method, itself. It is designed to help you and your team state, monitor and learn from their commitments to a course of causal action. It guides you to establish the results chain that drives the relationship between your deliverables and the outcomes of your projects or programmes. It consists of the following modules: DMF (sometimes referred to as Logframe), Resource editing, Project information, Export, and Printing. You can get the trial version here.

The AidProject M+E is a comprehensive and straightforward M&E software for Donor-funded aid projects. It empowers projects and programmes to employ good practices and satisfy your respective donor requirements. Basically, it helps the Project teams in: (a) Customizing the project via its Define Lookup Trees and Branches functions, customising include the DMF, Methods, Purposes, Gatherers (entities that will gather M&E data), Monitors, etc. The DMF/LogFrame will consist of the Impact, Outcome, Output, Activity – up to 10 levels) to define Indicators; (b) Define Performance Indicators together with their Locations and Periods and attaches additional criteria (your customised Trees & Branches) for reporting and analysis; (c) Maintain period targets and actual performance data (dates, text, money, yield, etc.); and (d) Distribute performance reports to Monitors (Donors, Government, Project Management, Stakeholders, etc.) and export spreadsheets to assist Gatherers.

One of the interesting features of the program is the Locations (Country, Constituency, District, Town, Village) for sub-dividing Indicators. It can report the progress of each indicators for every layer of the project. It has Purposes (relevance, effectiveness, efficiency, impact, sustainability) info that are used to define Indicators. The M+E Methods are pre-configured with 34 methods including Random Sampling, Stakeholder Analysis, etc. The Units of Measure (dates, text, money, tons, etc.) can have user-defined formats. It has a good organization of contacts to record the contact information of members of Gatherers and Monitor Groups. Finally, it has extensive text records consisting of Text Impact (High, Med, Low), Status (Open, Closed, Pending, Suspended), & Type (Achievement, Action, Risk, Issue, Lesson, etc.). A license to use AidProject M+E is absolutely free. You can download the program from here.

Package Planning (PACPLAN)

Solution is an e-governance tool for project and public sector management to improve its policy/program/project measurement that specifically targets gathering, collating, analyzing and disseminating policy/program/project information. It is easy to use software tools and processes combined with various graphics technology, internet technology and modeling tools such as economic analysis without stretching the organization capacity limitation. It enables project organization to implement, in the continuously changing environment, within the short time frame. It enhances management capacity through transparency, accountability, flexibility, adaptability, participation, predictability, continuity and supports the logical framework approach and facilitate stakeholders to appreciate a better M&E process where technology can later be enhanced.

Together with the software, PACPLAN offers a five-day workshop to help build your skill and learn from an experienced M&E system implementing practitioner. You will also be using the pre-developed M&E Software tools to fast track your learning and overcome obstacles in M&E system implementation, as well as ensure sustainability for continuing and improvement of M&E framework. You will become proficient at planning, design and development of M&E System as well as software tools, understanding stake holders and obtaining their consensus and handling difficult situations.

Posted by Ed Canela, guest Blogger

PIA and Decision Making for Development Results

admin — Wed, 25 Mar 2009 13:48:29 +0000

How can donors and partner countries assess the intended and unintended consequences of donor interventions? The Poverty-Network of the Organisation for Economic Cooperation and Development (OECD) proposes ex-ante Poverty Impact Assessment (PIA), which can assist in modifying the design of interventions to improve pro-poor impacts and identifying key areas for monitoring and evaluation (M&E). It can be applied to most modalities of donor support.

PIA is a process which helps policy-makers to understand the intended and unintended consequences of their interventions. This approach considers that good design of an intervention requires governments and their partners to understand the effect of their policies on diverse social groups, actors and institutions, including those not targeted by the policy.

The Paris Declaration on Aid Effectiveness stresses the importance of results-oriented frameworks, harmonisation and alignment to improve aid effectiveness and to assure better pro-poor outcomes. Yet, prior analysis of the impacts of policy and investment decisions on poverty reduction is a complex task. It is often built on contentious assumptions and is dependent on data availability. Ex ante PIA helps donors and their partners understand and maximise the poverty reducing impacts of their interventions.

It responds both to the need for accountability to partners’ constituencies and to the importance of transparent evidence-based decision-making. It can identify interventions with high impact on poverty reduction and pro-poor growth as well as mitigating measures to protect the poor. A broad application of ex ante PIA could also provide a basis for a harmonised reporting system on poverty impacts.

Poverty Impact Assessment helps decision makers determine strategic choices for public actions so as to have the greatest impact on reducing poverty and achieving pro-poor growth. PIA provides a better understanding about potential winners and losers of an intervention and thus strengthens a results-oriented approach. PIA helps to understand stakeholders and institutions that influence and are influenced by an intervention and
to understand the importance and inter-relationship of specific transmission channels through which changes are transmitted to the stakeholders assess the likely positive and negative outcomes for stakeholders taking into account multi-dimensionality of poverty assess the reliability of data/information and knowledge gaps.

Using PIA, policy-makers can estimate the likely quantitative and qualitative outcomes of the policy for poor groups, identify potential risks and assess the reliability of available data. Through involving people with different interests and approaches, ex-ante Impact Assessment helps to design interventions to be better targeted to achieve their goals and avoid unintended harmful consequences. Thus it also contributes to strengthening the transparency and accountability of democratically elected governments, and encourages consistency of policy-making across policy areas.

PIA is not just another new approach to assess the distributional impacts of interventions. It deliberately draws on existing approaches and their terminology, in particular on the Poverty and Social Impact Analysis (PSIA). While PSIA is more suitable for structural policy reforms and involves rigorous in depth analysis, including specific data collection for thorough social, political and economic analysis and comprising a whole range of quantitative and qualitative tools including micro and macroeconomic modeling. PIA is a ‘lighter’ approach, drawing predominantly on existing data and analyses. It provides an estimation of effects and a quick overview and focuses more on subsector or program levels. But it can also help at the initial phase of sector or policy reforms to identify requirements for a full-fledged PSIA. PIA is thus less resource demanding. While a complete PSIA requires more than 100 000 Euro, the estimated cost of PIA is less than 20 000 Euro.
See the PIA Concept Note: http://www.oecd.org/dataoecd/13/9/38878575.pdf
For more information on Poverty Impact Assessment and POVNET’s work:
http://www.oecd.org/dac/poverty
UNDP: Poverty in Focus 14: P(S)IA: http://www.ipc-undp.org/pub/IPCPovertyInFocus14.pdf

Originally published in http://tech.groups.yahoo.com/group/AGEG-eco/ Photo from www.ethicalleadership.org/programs/the-confluence