After learning how to design and plan a M.E.A.L. system, we can dive into the third phase of the M.E.A.L. cycle: data collection. Timely, high-quality data is in fact the foundation upon which your project team can measure progress, make decisions, and learn.
Data Quality Standards
Let’s start by pointing out that there are standards for defining various characteristics of high-quality data. Since the data you collect will never be perfect, it is important to determine, with the help of stakeholders, what quality and quantity are “good enough” for your decision-making (as well as learning and accountability needs). Regarding data quality, it is also important to consider the issue of bias, i.e., any systematic tendency or deviation from the true value. All data contain bias, and there are many types of biases that can affect the data. To reduce them, it is necessary to develop good data collection tools, valid samples, and strong data analysis. There are 5 data quality standards to consider:
- Validity: data is valid when it accurately represents what you intend to measure, i.e., your indicators. So make sure your data collection method helps you measure the indicators outlined in your PMP.
- Reliability: your data will be reliable when the collection methods used are stable and consistent, thus with methods that can be implemented the same way over and over again.
- Precision: data is accurate when it has a level of detail that gives you an accurate picture of what is happening
- Integrity: data are intact when they are free of errors that occur, consciously or unconsciously, when people collect and manage data, for example, if the questionnaire is not administered correctly or if data are entered incorrectly into the database
- Timeliness: finally, data should be available when you need it, meaning that if not collected on time, the data will not be useful to you in informing a decision. The timing of data collection plays an important role, which is why there is a dedicated timing column in the PMP.
How to design a Data Collection Tools
As you begin to design your data collection, ask yourself, “what do I need to know?”. The answer will inform the development of your instruments and the sampling process you will use (remember that some of this work has already been done in the PMP when you selected the indicators). A data collection tool consists of 3 components: introduction, questions, conclusion.
In the introduction, you will explain the project, cover the data collection process and the ethical principles that guide your collection. Obviously, this information should be shared with respondents prior to data collection. In the introduction, you will need to know what the information collected is for, how the respondents were identified, how long the collection will take, how the data will be used, and who will have access to the information collected. Based on this, you will also need to explain before the data collection what ethical principles are guiding you: according to the principle of informed consent, participants should understand the project and understand that their participation is always voluntary, so they can leave at any time. The existence of security plans to maintain the confidentiality and anonymity of the data collected, and finally what plans are in place to share the results with participants.
The second step is to list the questions to ask the respondent. The questions should be designed to collect the data you need to meet your information requirements, so there are guidelines for this:
- Invest time in the layout and design of your data collection tool, even if it seems trivial, but this way your tool will look professional (layout, style, graphics) and be easier to use;
- Make sure the language you use in your questions is simple, clear, and appropriate for the context;
- Organize your questions using a clear and orderly sequence. The structure of the questions asked should be logical and make sense to data collectors and respondents;
- Make sure your data collection tool includes fields to record important data analysis and management information, such as the day and place of data collection, the respondent’s gender, age, and geographic origin, and, if necessary, a number or code assigned to each participant
Finally, always close an instrument in the conclusion by offering respondents the opportunity to ask questions and provide feedback on the experience. Also, it is a good idea to thank respondents for their time and reiterate how the data will be used and when respondents will be able to know the results
Now let’s see the quantitative and qualitative methods and their tools to conduct the survey.
MODULE 6: Collecting M.E.A.L. Data – Download
Find the previous modules here.
If you are interested in knowing more about project writing and evaluation and would like to have the assistance of professionals, you can email us at ssr@signis.net. At SIGNIS Services Rome we are experts in the sector and have been involved in project writing for the creation and development of communications projects all over the world for decades.
*This content was curated by Valeria Appolloni and inspired by the materials published on Kaya, published for non-commercial and educational use.