FAQ about Monitoring and Evaluation

Self-Service Analytica
Frequently Asked Questions
about Monitoring and Evaluation

Explore answers to the most commonly asked questions about monitoring and evaluation

Your questions answered

We have put together some of the common questions we received during the inaugural Synergy M&E Webinar Series. The webinars looked at key challenges to creating a culture of monitoring and evaluation in organizations as well as the trends and important innovations at the intersection of M&E and information technology. We hope these answers will give you more information about monitoring and evaluation in government and nonprofit organizations.

We continuously work on creating new content for the section, so please check back regularly for the latest updates.

Webinar 1: Current State and Key Challenges

What challenges do you face while working with organizations that are new to M&E?

Though the leadership oftentimes agrees with the need for a monitoring system or a method for evaluating the efficacy of a project, program, or organization performance overall, in practice designing and implementing a new monitoring system has several challenges, such as:
• stakeholder’s buy-in,
• technology shortfalls,
• lack of experience,
• limited financial and staff resources,
• gaps in technical knowledge with regard to logical frameworks design and definition, performance indicators definition, the retrieval, collection, preparation and interpretation of data,
• inefficient monitoring and evaluation practices,
• timelines.

Question from: Jaya

Accountability is the main objective of M&E in nonprofit organizations. How can we link M&E and accountability? Is there any process or procedure?

Accountability is one of the main goals in M&E, both in government and nonprofit organizations. M&E for accountability commonly focuses on upward accountability to a government or funding agency. It is often an obligation to demonstrate that the contracted work has been conducted in compliance with the agreed standards or to report on results and plans. Downward accountability involves making accounts and plans transparent to the primary stakeholders, i.e. clients.
With M&E software platforms such as Indicata, it is possible to demonstrate whether the contracted work has been accomplished in accordance with the agreed plans or not. It is very important to have tools which will help report results vs plans and find gaps or inconsistency for further fixes and performance improvements.

Question from: Somar Rezk

What methods should be used to ensure data quality in a project?

The first and most important thing is to provide 'smart' data entry templates and train the staff for effective data collection procedures and rules. Digital data gathering with validation in place will help omit most of the errors. Next should come data cleaning exercises. Data Quality Review (DQR) and Data Quality Audit (DQA) tools allow to evaluate data quality across projects and/or programs. Synergy Indicata provides tools to review and measure the data quality in projects.

Question from: Moseed AL-Arasi, Nilaramba

What tools can be used to drive and document learning in an organization?

Learning is the process through which the information generated from M&E is reflected upon and intentionally used to continuously improve a project’s ability to achieve results. Documenting targets, results, changes and challenges can serve as input from past learning into the new cycle of a project or program. There are several ways/tools that can help you visually represent your goals, outcomes, outputs, activities, related indicators and document the learning that can be saved as a knowledge base. Synergy Indicata provides all the necessary functionality for that.

Question from: Anita Ubah

I think it is important for people to understand that M&E is used to identify and mitigate issues that may impact a project and not to evaluate the performance of individuals in a given project/organization.

It can actually do both, depending on what the organization wants to focus on, but in the end, the goal is to find the bottleneck and improve processes and performance.

Question from: Andy Chi Tembon

The current COVID-19 pandemic's impact on business continuity has underscored the dire need to bridge the digital divide, especially for organizations working with resource-poor and remotely located communities. What experiences and guidance do you have regarding cost-effective technologies and methods for on-going monitoring and reporting of programming?

When operating in poor and remote locations during Covid-19, there may be the need to pause non-critical M&E for some projects while maintaining essential or time-sensitive functions. The importance of phones, internet access and personal protective equipment (if in-person data collection is necessary) is essential. Data collection shifts to smartphones and tablets.

Synergy Indicata encompasses a data collection platform that allows offline data collection, conducting surveys offline and synchronizing it with the server when there is the Internet.

Question from: Kombo Choga

What master's degree would you recommend to someone who has been practicing M&E for a while and is very passionate about it?

Follow your passion, focus on where your interests are, for example having a degree in International Development or in Social Development of Communities would help as well.
These universities offer master's, PhD degrees and different certificates either directly in evaluation or with available concentrations in evaluation.

Question from: Ruth Amamo

Do you have any experience in practicing outcome harvesting in the non-governmental sector? Would you recommend this method?

Outcome evaluation can be done based on both its outputs and independently through outcome level indicators. Outcome harvesting has proven to be a useful approach in a variety of locales and for diverse interventions. It is used when the focus is more on outcomes rather than on outputs or activities, and is well-suited for evaluation in dynamic, uncertain situations, for example when the outcomes were not specific or measurable at the time of planning the intervention.

Question from: Anastasia Kushleyko

As global mobility is limited due to COVID-19, how do you conduct distance monitoring and how do you verify the information/data?

It is possible to use remote tools such as surveys/interviews and other data collection methods and where possible rely on indicators that can automatically gather data in real-time, e.g. IoT data collection nowadays is quite modern. Please watch our second webinar to learn more.

Question from: Suman Gupta

How can we establish an M&E culture in organizations?

It is very important to understand the importance and need of an M&E culture in the organization. Thus, to start the M&E transformation the following actions should be performed:
1. Engage the management as their ownership in this initiative is crucial.
2. Prepare surface-level capabilities (skills and knowledge) within the organization, if there is no any, then hire SMEs (subject matter experts).
3. Be ready for changes in deep-level capabilities such as mindsets - people are resistant to changes, so management should find ways to have a smooth transition, by involving employees and gaining their ownership, or making it clear that M&E is not for rebuke but for a better way of working, for finding gaps, and for making programs as effective as possible.
4. Produce tangible 'artefacts' - such as a visual representation of a system to be demonstrated during monthly/quarterly all-staff meetings, or semiannual M&E reports/dashboards to serve as employees' homepage.
5. After having all these in place - it will need automation, thus technology is playing an important role for real-time data collection, ease of key findings, harmonisation and automation of business processes.

Question from: Liliana Carral, Jaya

Why do some organizations see M&E as a donor requirement but not as integral part of the implementation?

Probably they see M&E for rebuke or for blaming accountable people, or they have never been shown the advantages of M&E. Please check the relevant question raised by Liliana Carral and respective answer.

Question from: Maazua

Some M&E practitioners say that it is important to collect much data to use in the future, the panelists say we only have to collect the data we need. Are the M&E practitioners mentioned above wrong?

We would not say that other M&E practitioners are wrong. They would document the data, however, they would use only the data they need, so the other data will become an asset only when they realize what and why is needed to be analyzed. Data gathering and cleaning requires effort, so it is a matter of resources and cost - can we afford to gather and ensure the quality of data that we do not need but may need in the future? Also what if the collected data will become not actual anymore?

Question from: Aster

How can you convince the senior management to involve you in the planning level as an M&E officer?

We recommend preparing a robust and solid M&E plan including a result framework and indicators that will allow measuring the success of the given result framework. You need to make sure that the result framework is focused on your organization's global strategic plan and this must be aligned with your management’s vision. The whole M&E plan must be feasible, implementable, in line with your corporate plans, and result-oriented. Then a very good presentation of this will help you become part of corporate plans.

Question from: Shorooq Shabillo

How can we help the local government and community conduct M&E for humanitarian and development organizations using an information system? How to make organizations accountable for the communities they work for? Are there other successful stories like charity.org?

This is a very global question and we think that the starting point can be the active collaboration with the local government or communities. In these processes, M&E specialists and consultants need to play the key role, to convince and prove the real need of M&E science in spheres like education, health, child protection, etc. As soon as the governing bodies agree on the importance of M&E, you can initiate the next step, which is the design of an M&E Information System, and then translate all this knowledge into technology. All these should happen at a contract level, so accountability will be an obligation. At a contract level, the goals for implanting organization should be stated so the progress will be possible to track.

We are proud to highlight that through Synergy Indicata, complete monitoring and evaluation software, which streamlines complex M&E processes, we have provided solutions to several governments and international organizations, such Save the Children, UNICEF, Educate Above All, etc. For more information about our projects please check here.

Question from: Hanomy Hajeb

Because of the present COVID-19 pandemic we can't go to the field and do the baseline survey we have planned. What can be the alternative strategies to do the baseline with minimum field visits or without field visits at all?

Based on the experience that we have with our clients, first of all, you need to adapt your Result Framework and Indicators to correspond to the current situation based on the COVID-19 pandemic. You might need to cut costs from non-priority objectives (during the period of the COVID-19 pandemic) and instead mobilize your resources on hiring local partners, acquiring tablets or mobile applications for data collection, etc.
In a data capturing process, you should involve a local partner to minimize the number of on-ground operations, then there might be a need for data validation as well (again by another local partner).
Developing/purchasing a mobile application for field visit surveys can be useful. Another option is online surveys with a layer of random data quality checks.

Question from: T. Senthoran

How do M&E experts ensure that local people get involved in data collection?

The target local people should be registered providing specific contact information. For data verification, random calls will be made to ensure that the data collector didn’t cheat and to avoid any planned responders.

Question from: Eniola Saheed Adedayo

I'm a socio-cultural anthropologist who deals mainly with collecting and analyzing qualitative data with social research methods. It would be really helpful if the speakers could speak more about their favorite software for the analysis of quantitative data.

As representatives from Synergy, we would like to specify Synergy Indicata, which is an M&E software system for storing both quantitative and qualitative data. One of its advantages is its sophisticated and powerful reporting tools (analytics). Indicata’s robust reporting provides users more power and flexibility in the creation of ad-hoc reports, and their export to Word, Excel and PDF. Reports by any combination of groups, columns, rows and quantitative data fields can be created by non-technical staff on-the-fly in a highly user-friendly and logical manner with the Report Builder.

Indicata provides Data Entry form builder where any type of quantitative fields can be stored for further surveys/data collection.

Question from: Mara Tissera Luna (Argentina, Hungary)

How can I address data collection during an emergency? While my program did not necessarily target a relief situation, an emergency came up.

The nature of most emergency responses demands a more relaxed and sympathetic M&E approach. Immediate response comes first, and any M&E / data collection is often secondary. That said, emergency response activities have to be planned, executed, and monitored – the extent of which is guided by parameters of the response.

During emergency, only primary needs should be collected, or only basic and most important indicators should be considered (e.g., number of people injured, or number of people saved, etc.).

Also, data collectors should come up with rapid methods for data collection, for example, if during earthquake people should be registered then it might happen by taking person’s fingerprint - just for identification, but then in post factum detailed level information can be collected and monitored.

Question from: Ahmad Ramzi

Do you have any suggestions for conducting a program evaluation that is inclusive of changes due to the COVID-19 pandemic?

You need to tailor your program strategy plans, result framework, and indicators so that they will reflect the current situations and will be feasible and practice for this period of time, e.g. define new simple (minimum level of disaggregate, no need for baseline, etc.) indicators to measure the results. Afterwards via composite indicators, you can combine data of these simple COVID-related indicators with the regular ones. Please also check the answer of T. Senthoran's question above.

Question from: Samantha Woodcroft

How can an effective M&E system be developed in organizations other than project implementation? How can the best indicators be set for their organizational M&E only?

You need to design corporate-level result framework (e.g. including results/objectives relevant for partners’ performance as well). Then you should define your key performance indicators not only at an output level but also at an impact level. For example, to measure partners’ performance the following simple KPIs can be defined and attached to the corresponding result level:

  • Revenue through partners
  • Number of partners
  • Number of deal registrations
  • Number of people trained by an implementing partner

Based on the sector, you may have other indicators, e.g. for education - “How many out of school children have been reached out”, or for health - “How many children have been vaccinated”.

Please check the answer to the relevant question raised by Liliana Carral and Jaya.

Question from: Saba Khan

We are all aware that data is a very important aspect of M&E and it anchors decision making. How best can big data analytics be utilized in M&E to improve decision making?

We already have experience with implementing AI solutions within M&E systems to give our users smart predictions.

As a very simple example: if the system noticed the correlation between two indicators: “Car accidents mortality rate” and “Number of traffic violations”, then AI can predict increasing the ticket cost for traffic violations as an effective solution to the problem, so by establishing a liaison between indicators and analyzing them AI can identify and close the gaps.

Question from: Dexter Hati

What are the ways of monitoring and evaluating results of projects on social cohesion among refugees and host communities?

As a relevant example for you, we will mention the system developed through Synergy Indicata “The European Union Facility for Refugees in Turkey”.

The Facility Monitoring Platform (EU FMP) is an automated online monitoring and evaluation system which was gathering mainly data for Syrian refugees. The main objective of the system is to support the Government of Turkey in formulating a comprehensive strategy, effectively managing implementation, and promoting the accountable and transparent use of resources. This kind of systems can dramatically change the precise identification of beneficiaries and their further progress. The first step should be the proper identification (via ID, fingerprint, etc.) of beneficiaries and then their involvement in the right program/project. Pre- and post-evaluation should be there for assessing results accurately. These will help you measure the impact of the programs and/or projects.

Question from: Deniz Menteşeoğlu

Ho do the panelists see the application of big data in M&E?

This was addressed during the webinar by Emilya Khachatryan – please check the recording:

Cheap mobile devices and reliable internet services are enabling organizations to collect larger volumes of more reliable data.

Nowadays most of big data is collected from IoT devices such as sensors, cameras, smart parking solution, GPS tracking devices that are embedded in the water and power infrastructure, for example.

Data of “Electricity consumption rate” or “Basic Water Supply” indicators can easily be conducted through these devices, which will bring to big data. M&E is nicer and more accurate whenever data is big, i.e. better reports and analysis can be conducted, but in this case, M&E software becomes a must.

Potential problems include:

  1. M&E practitioners are inexperienced with large datasets and complex data models.
  2. Data cleanup and aggregation can be difficult.

Question from: Anush Shakhsuvaryan

The panelists started by saying that the key issue for M&E was to be useful for ongoing project activities and management - that is a different priority than accountability. Do you think one is more important?

Both, of course, are important, and we can’t really say which one more as they are closely intertwined. Accountability is necessary but having M&E on top of it you will have both necessity and sufficiency. M&E can be a useful tool during project execution if the parameters allow it to be, e.g., frequency of data collection. M&E can ensure better accountability both towards donors, as well as clients/beneficiaries.

Question from: Charles Downs

What tool is the best to analyze qualitative data?

Please check the answer to the question by Mara Tissera Luna posted above.
Question from: Gulnara Kokonbaeva

What is the best way you can collect feedback from beneficiaries especially during the COVID-19 pandemic? Is there any structured or recommended tool which can be used in a fragile context like Somalia?

Face to face meetings are probably the best way to collect feedback from beneficiaries as they can also facilitate identification, validation and deduplication. That said, face to face meetings are not always possible, due to the lack of funding, limited access to rural communities, conflict or a pandemic. In case of sufficient access to the Internet, beneficiary feedback can be collected online, however as we know this is mostly not possible, and in most cases the only alternative is using phone calls, SMS or IVR, or by using local focal points.

Recommended read: Data Collection in Times of Physical Distancing

Question from: Mubarak Nour

What are the properties of a good M&E system?

First of all, a good M&E system should be feasible and implementable. There should be SMEs who will be a dedicated staff in M&E-related activities. These team should define and design M&E work plan with corresponding budget, M&E frameworks/Logical Framework, smart KPIs, etc.

Recommended read: 12 Key Components of M&E System

Question from: bbanguna

Have you come across biases in data gathered during COVID-19?

COVID-19 has had a negative impact on data collection and might have emphasized certain biases.

Recommended reads:

Check your bias, gender data experts warn

COVID-19: Guidance for labour statistics data collection

Question from: Dalita Artin Avanesian

How can you promote monitoring while you have a PMO, i.e. most of the PMO job is about following up and monitoring?

The primary function of the Project Management Office is to define and maintain standards for project management within the organization. There are several types of PMOs - Supportive, Controlling and Directive. Centralized and corporate M&E system approach is dependent on the type of the organization and it should be aligned with the management and PMO strategies and objectives.

If the question is about a project-level M&E, monitoring and control (please check PMBOK 6 edition) is one of the main phases (out of 5 phases - initiation, planning, execution, monitoring and control, closing) of project life cycle which should happen throughout the entire length of a project, and it should be conducted according to the standards set by PMO.

Question from: Hamza Abdalla

Is it better to have a centralized M&E unit or specific and decentralized units especially in the public health sector? People in this industry don’t think that M&E should be a centralized unit.

Having or not having a centralized M&E unit depends on the structure of the organization and/or sector, its objectives, as well as on the M&E practice and culture.

While there are benefits to centralizing M&E, not every organization is able to do so. Reasons may lie in the non-compatibility of programs, which can require various, sometimes incompatible M&E approaches, frameworks, subject matter expertise, etc.

Question from: Hamza Abdalla

Knowledge management is often linked with M&E, and where do planning and implementation of projects stand there?

Project knowledge management – cost, schedule, scope, resources, stakeholder, and other knowledge management areas are fulfilled in 5 phases - initiation, planning, execution, monitoring and control, closing correspondingly. M&E or Monitoring and Control (check PMBOK 6th edition) is an important part of any project and it should be implemented throughout the entire duration of a project.

Question from: Fikru Habte

How can you overcome the challenges of M&E knowledge and capacities within local organizations?

First of all, you need to identify stakeholders and other things that are creating those challenges. Then, you will need to gain the ownership and interest of those identified stakeholders, trying to find some incentive and motivation for each. You will need to demonstrate to them that good M&E is not for finding guilty people or for rebuke. You should make clear to everybody that good M&E is for a better way of working, for finding gaps, for making programs as effective as possible.

Please also check the answer to the relevant question raised by Liliana Carral and Jaya.

Question from: Elisa Riquelme

Where would you suggest that the M&E team seat in an organization?

Our recommendation would be to establish and have a separate independent department.

Question from: Lydia Mukaye

How can an M&E officer work more proactively to create an appetite for M&E's work within the organization?

Please check the answer to the question raised by Liliana Carral and Jaya.

Question from: Ahmad Ullah Qazi

What methods can we use to monitor humanitarian assistance in emergency?

For humanitarian assistance in an emergency, one of the most important activities is the identification of victims, then further tracking of corresponding beneficiaries through management and information systems. During emergency people can’t provide an ID card (passport, UNHCR passport), thus rapid biometric identification (fingering, iris scan, etc.) should be used. Further, if the beneficiary gives consent, details should be entered in a management system for detailed monitoring. Note, personal data is sensitive, thus all governmental, GDPR rules should be considered during this activity, i.e. in case beneficiary is requesting to delete their personal data from the system, that should be done immediately. Another method is data anonymization/encryption that should be carried out in Management and Information Systems.

Question from: Salauddin Ahmed

Which M&E skills may be considered trending?

This may vary. The trending skills now may include: data collection automation (through IoT and smart devices), calculation of results (Impact, Outcome, Output, etc.) and aggregation automation through a management and information system (M&E systems), AI integration for projections.

Question from: Mohammedhal

Webinar 2: Technologies and Innovations

Are organizations ready to cope with the change in terms of the availability of a budget and trained staff?

It depends on the organization and type of the project/program. However, we see a trend towards the increase of M&E budgets and relevant staff training (excluding the COVID-19 period).

Question from: Kelly-Ange Irakoze

How will 5G technology affect the way we do M&E?

We expect that 5G technology will improve the Internet access for a large number of people, therefore enhancing data collection and M&E data reporting on-the-go via mobile phones, tablets and other portable devices.

Question from: Pasteur R.

How can we compromise between addressing the increased demand for real-time data and the need to provide robust and technical sound indicators?

The use of technology allows M&E specialists to rapidly translate and structure real-time data into indicators and create a results framework/theory of change/any other type of frameworks to organize, systemize and analyze the given data.

Question from: Ashraf

How can M&E make an impact on the project implementation?

As you plan your project objectives, logistics and resources, it is best to start designing and incorporating your M&E simultaneously with the input and feedback from all stakeholders and team members involved in the project. It is important to link your M&E plan with the project’s strategic plan in an integral manner. This ensures an effective allocation of resources, establishment of roles and responsibilities, an understanding of the political and administrative structures of the community where the project will take place, etc. Likewise, planning the M&E activities early on also helps identify your baseline values to measure the success and failure of the interventions during the project implementation and your impact during project implementation and closing.

Question from: Maazua

How can we ensure that everyone in the organization is made aware of the M&E frameworks in place so that they are involved in the process to ensure their impact on the overall objectives is felt?

You should communicate the importance of M&E in your organization to the stakeholders, as well as avoid common mistakes in M&E, which reduce the effectiveness of M&E data collection and reporting. Make sure the M&E data is communicated back to the stakeholders who report the data so that they are aware of its importance in the big picture.

An M&E framework and its progress can be part of every employee’s homepage.

Question from: Nelly Saiti

How does M&E contribute to ensuring that qualitative data is quantifiable and what technologies would you recommend?

A common method that evaluators use to analyze qualitative data is triangulation, which involves taking data, finding themes, coding them, and then comparing or triangulating the data from different data sources and different data collection methods. Here automation or AI technologies can be quite helpful.

Question from: Nelly Saiti

How is the development or program effectiveness measured at global/HQ level of an organization working in different countries and with different donors? What M&E systems do you use and how do you standardize them?

A global/HQ level Result Framework can be maintained and individual Project or Program level framework results can be mapped to the global result framework results. This practice can be done via Indicators and Composite Indicators (through the bottom-up aggregation) at the global level or through the mapping of result levels to maintain a cohesive and connected M&E structure. This way you will be able to see how each project/program is contributing to the global corporate level.

Question from: Retta G.

Since outcomes are typically more difficult to monitor and evaluate and data is often not available and primary data collection is required, what tools can be used to manage and collect the outcome data?

Outcome data can be collected through:

  • Surveys
  • Interviews
  • Focus and other small groups
  • Observations
  • Records and documents

Question from: Hamda Hersi

The topic of remote monitoring is very critical for many areas. What impact will this have on the technology?

With the use of technology, remote monitoring is improved on all fronts, as it is possible to automate the following key concepts which remote monitoring hinges on:

  • Include M&E as part of ongoing practice,
  • Standardize and simplify,
  • Open opportunities for replication; recognize good practice,
  • Achieve participation from users,
  • Focus on demand-side requirements; aim for quality engagement in M&E processes and obtain multiple perspectives,
  • Seek out lessons for improved practice,
  • Budget and plan for adequate time and resources – including training where necessary,
  • Include high-level (long-term) goals.

Question from: Matthias Leibbrand

How can M&E help avoid data overload and still be effective in measuring impact?

A properly designed M&E provides a structured and systemized option of sorting, organizing and maintaining data via indicators and M&E frameworks, which allows M&E specialists to sort and maintain essential data, without getting overwhelmed by data. In M&E software solutions, the bulk core (sometimes even unstructured) data is segregated in different databases while only essential and aggregated data is kept in the main system database.

Question from: Maazua

Online data collection or remote data collection is cost-effective. How important are observation and face-to-face interviews to monitor the effectiveness of the intervention?

Yes, observations and interviews are also important tools for data collection, especially for outcome results. However, the data collected via those methods also need to be digitized and systemized to maintain data completeness and allow M&E specialists to analyze it effectively.

Question from: Anu Bhowmik

Which is more useful to consider in M&E - output or outcome impact?

All components are required to structure the results framework of a program/project. It is important for M&E practitioners to design the framework in a way that covers the main goal of the program/project but for ease of implementation and measuring its impact this goal should be broken down into smaller goals (outcomes), then to be broken into feasible and tangible outputs.

Question from: Yaghouba Kane

In terms of technology, how can we have a successful MIS system (a successful automation of M&E system) that can help track routine data (output data), capture data coming from the field, aggregate it to measure indicators (with MIS in built business logics) and generate reports or dashboard in real-time for the program team and management?

For any technology project, there are three paths – open source, custom development and proprietary product. Each approach has its own pros and cons and the decision depends on many factors (organization’s vision, budget, timeline, maintainability, etc.).

Based on our experience, organizations that have chosen custom development usually carry a higher risk of failure. Also, these organizations tend to switch to open source or proprietary product once they realize that their in-house product either is not capable to scale or requires high maintenance in comparison with other similar products.

Please read this whitepaper to learn more about open-source vs. propriety product.

Please also check the answer to the relevant question raised by Liliana Carral and Jaya.

Question from: Retta G.

How can we keep data secure if we introduce an M&E technology in an organization?

For sensitive data protection, M&E software providers must develop a data encryption policy based on which they should be able to store and protect data with different types of encryption algorithms. Synergy has developed a GDPR- compliant policy. For example, in Synergy Indicata a user can define which data is sensitive, and based on that the system will automatically create a database table with a column-level encryption.

Also, proper M&E software solutions have multi-factor authentication and multi-layer data encryptions, which limit unauthorized accesses and cyber-attacks. In addition, most of the popular cloud providers cover all known cyber-attacks from an infrastructure perspective.

Question from: Badrun Nahar

Do you think media monitoring could help address the M&E challenge in the current pandemic situation?

Yes, any type of remote and automated data collection that is feasible and useful addresses ME challenges during the current coronavirus pandemic.

Question from: Badrun Nahar

How often primary data collection is recommended for outcome level indicators and what tools and systems are available and affordable?

Usually primary data collection is recommended for output level indicators while for outcome level indicators surveys are recommended tools.

Synergy Indicata is a powerful M&E data entry (for both qualitative and quantitative data) and analytics tool.

Question from: Sinshaw

How do you measure the sincerity of the responses? When talking to people, we are trained to see if they have doubts about what they are saying. How to capture this in remote data collection?

First of all, the questionnaire template should be tailored to address the challenges of remote surveys, i.e. to minimize the questions whose responses are dependent on person's sincerity/emotions (to have structured answers, e.g. Yes, No or to provide predetermined options).

Statistics shows that if in about 10% cases a respondent is changing their answer, there is a possibility of insincerity or non-confidence. In these cases, double verification questions can come for help.

Question from: Adiza Lamien Ouando

Is it necessary to collect huge data to measure outcome? I think we can use LQAS Sampling techniques to measure the outcome.

This totally depends on the outcome description, and outcome result can be measured through surveys, interviews, focus groups or observations, without collecting big data.

The LQAS random sampling methodology is mostly for quality control, which provides less information but often requires substantially smaller sample sizes. As for huge data - if it is collected correctly and accurately, it is always possible to avoid mess and manage it through technology, i.e. automate, aggregate, slice and dice, and filter big data.

Question from: Anu Bhowmik

How does the technological advancement affect the M&E activities?

Some technological advancements like NLG (Natural-Language Generation), AI (Artificial Intelligence), ML (Machine Learning) and IoT (Internet of Things) can shift the efforts and focus of M&E practitioners towards value-add activities and reduce their routine burden, specifically on data collection and reporting activities.

Question from: Mubarak Nour

Please recommend digital survey tools/platforms?

The recommended standalone platforms for conducting surveys include SurveyCTO, KoBoToolbox and SurveyMonkey. They are robust platforms with the ability to create complex data collections forms and integration with data analysis platforms.

Also, M&E solutions like Synergy Indicata has the built-in survey module. This works seamlessly with other M&E-related functionalities while data visualization and analysis can be done within the solution itself.

Question from: Sinshaw

Does changing to Industrial 4.0 affect M&E much?

Absolutely! This means that cutting-edge technologies like Predictive Analytics, ML (Machine Learning) and IoT (Internet of things) will be engaged heavily to help organizations deliver their programs in the most effective and efficient manner. The vision for M&E might be to have machines fully/mostly cover the monitoring component and assist M&E professionals/program directors in driving and leading mission-critical programs.

Question from: Khin Zaw

How about RCTs type if evaluation designs and donors expect that? Often budgets constrain us to conduct such powerful surveys.

A randomized controlled trial (RCT) is an experimental form of impact evaluation in which the population receiving the program (direct beneficiaries) is chosen at random from the eligible population, and a control group is also chosen at random from the same eligible population. In this case surveys and data collections are more cost-effective, thus donors should agree with this approach if the need of this evaluation is justified in the scope of the project and all templates, processes are designed/planned correctly.

Question from: Sinshaw

Is it recommended to use LQAS to track outcome indicators and what merging of anticipated change in indicator progress is acceptable in that regard?

Lot Quality Assurance Sampling (LQAS) is a random sampling method used for quality control in industrial production. Compared to similar sampling techniques like stratified and cluster sampling, LQAS provides less information but often requires substantially smaller sample sizes. It uses small sample sizes to classify health or administrative geographical areas, to inform if these areas have achieved a pre-determined target for a given indicator (measuring outcome indicators).

Conducting population-based surveys using LQAS for both baseline and ongoing M&E assessments for outcome level results can be very useful. LQAS is a tool for managers to help them improve the overall program performance.

Is Re indicator progress acceptable or not? If this was anticipated, benchmarks, targets should be set up correspondingly. Keep in mind that M&E should always be feasible.

Recommended read: Lot Quality Assurance Sampling Measures Performance

Question from: Sinshaw