Monitoring and Evaluation of Health Promotion Intervention

       Monitoring and Evaluation of Health
Promotion Intervention

Monitoring and Evaluation
.
Monitoring

• A systematic and continuous assessment of the progress of an activity over time
• It is a part of implementation
• It can be done through the process of collecting, coordinating, processing, measuring and communicating information to assist management to make decisions
• It involves follow up of:
o Inputs (funds, personnel and others)
o Process (such as activities/tasks being done according to planned interventions)
Evaluation
• The systematic collection, analysis, and reporting of information about a program in order to improve planning or implementation of current and future program activities.
• It includes areas of both process and impact to assess whether the set out objectives have been achieved.
   This can be internal (by implementers) or external (by outsiders).
• It is done before implementation to assess development needs and potentials as well as feasibility of the plan.
• It is also done during implementation (formative evaluation) to identify areas for changes or modifications.
• It detects deficiencies and immediate redesign of intervention strategies.
• It is done at the end of intervention (summative evaluation) to assess project or programme effect, outcomes and aim at obtaining information on:
o Effectiveness of the programme in achieving its stated objectives.
o Its contribution to developmental goals.
o Efficiency of the programme or project on utilization of resources.
o Sustainability of the project results.
o Whether to continue, modify or terminate the project.

Importance of Monitoring and Evaluation
The Importance of Monitoring
• To make sure that work progresses according to schedule
• To ensure that agreed upon rules and regulations are maintained
• Resources are used rationally as planned
• Required information is available and used
• Detection of problems during implementation and to take corrective action
• Verification whether plans are being implemented accordingly
• There are many tools which can be used to monitor health intervention, among them are:
o Plan of action
o Quarterly technical and financial reports on the implementation of the intervention
o Supervision reports
o Activity progress reports.
o Outputs/outcomes (are the intended objectives and expectations met?)

The Importance of Evaluation
• To determine performance effectiveness and efficiency of services/or program which include:
o Decision whether an activity is worth doing
o Determine whether the objective set were achieved
o Identify ways of improving a program, such as ensuring that all activities are relevant and appropriate to the health needs of community members, and removing potential barriers to participation
o    Determine (formative evaluation) whether activities should be continued or not
o Use of the evaluation information for a range of purposes, including promotion,
fundraising, attracting potential partners, and lobbying and advocating for policy changes to create healthier communities
o Determine the project should be extended else where
o Justifying the program to management
o Providing evidence of success or the need for additional resources
o Increasing organizational understanding of and support for health promotion
o Encouraging ongoing cooperative ventures with other organization.

Evaluation Process
• Produce useful feedback for program refinement
• Provide feedback to a funder on how resources were expended
• Determine program activities’ success rates
• Document successful processes so they can be repeated in the future
• Demonstrate program activity to the media or community even before outcomes have been attended

Methods Collecting Data for Evaluation
• Depending on the evaluation purpose, evaluations can use qualitative or quantitative data collection methods or a combination of both
• The selection of data collection methods depends on a number of factors, including:
o The purpose of your evaluation
o The questions the evaluation is seeking to answer
o Financial resources
o Time and skills
• If you want to explore participants’ or workers’ experiences, it would be more appropriate to use qualitative methods.     

This will allow you to ask ‘how’ and ‘why’.
• If you want to measure the numbers of participants in a program or measure the degree of change in a health measure (such as body mass index) or behavior (such as participation in physical activity), then it may be most appropriate to use quantitative methods
• Quantitative methods answer who, what, where, and how much
o They target larger numbers of people and are more structured and standardized (meaning that the same procedure is used with each person) than qualitative methods.
• Qualitative methods answer why and how and usually involve talking to or observing people.
• When using qualitative methods it is best to use several approaches. This concept is often referred to as ‘triangulation’ and using it strengthens your evaluation.

   The Process of Evaluation

Key Steps In Evaluating Any Health Program (HP)
• Clarify your program
o What underlying needs and resources must be addressed?
o Goals, what are the goals, target population, and desired outcome objectives?
o What should the members of the intended population think, feel, or do as a result of the health promotion program in contrast to what they thought, felt, or did before?
• Define program activities and outputs
o How can these changes be measured? Establish measurable program indicators
o Input indicators: measures the resources that are used in the provision of services e.g. human, materials and finances
o Output indicators: measures the output following a process resulting from certain inputs e.g. community trained, number of latrines constructed, number of health education sessions conducted
o Outcome indicators: measures result following some activity e.g. number of people came for counselling and testing services.
o Impact indicators: Show the long term effects of an intervention e.g. reduced number of malnutrition in the community, leading to improved health status of children in a given time
o Engage stakeholders/community members in the HP program
o Understand stakeholder interests and expectations (needs assessment)
• Assess resources for the evaluation
• Determine availability of staff and resources
• Determine amount of money allocated for evaluation
o Conduct a resource mapping and asset assessment
o Examine the community resources and other assets that exist (or do not exist) to respond to the targeted problem/issue in the community
• Design the evaluation
o Select type of evaluation to be conducted
o Design evaluation framework
o Consider ethical and confidentiality issues
        Develop evaluation questions (based on program goals and objectives and stakeholder interests and expectations)
      Select a target area to be assessed. Be specific in defining the target area so you can remain focused on the types of data to collect
        Gather data to develop a clear ‘picture’ of the nature and extent of the problems in that geographic area
        Gather data that help describe the nature and causes of the problem. Examine all data sources that provide information on the problem including contributing
        Assess the risk and protective factors of participants in the target area
        Once you have identified a target group, conduct a systematic assessment of those risk conditions that contribute to the problem/issues and those protective factors that improve risk conditions
• Determine appropriate methods of measurement and procedures
o Select the most appropriate qualitative and quantitative methods for collecting
information about your program.
o Consider qualitative versus quantitative methods; and
o Select your sampling design.
• Develop work plan, budget and timeline for evaluation
• Collect the data using agreed-upon methods and procedures
o Pilot test
o Determine data collection techniques
o Use tips for data collection
• Process and analyze of data
o Prepare the data for analysis
o Analyze the data
• Interpret and disseminate the results
o Interpret the results
o Present the results
o Share the results
o Take action

Selecting Evaluation Type for HP Programme
• When selecting types of evaluation for your HP program, it is important to consider:
o The length of time the program has been in operation
o Your program’s stage of development
o What, if any, types of evaluation have already been conducted
o The amount of resources you have to devote to evaluation
o The interests of your program stakeholders
o How do you expect change to occur?
    Will it be slow or rapid?
    What measurable intermediate outcomes (steps toward the desired behaviour) are likely to take place before the behaviour change can occur?
    The behaviour change map you created in should provide the answers to these
questions.

o How long will the program last? What kinds of changes can we expect in that time period (e.g., attitudinal, awareness, behaviour, policy changes)?
     Sometimes, programs will not be in place long enough for objectives to be met when outcomes are measured (e.g., outcomes measured yearly over a 5-year program).
     To help ensure that you identify important indicators of change, decide which changes could reasonably occur from year to year.
o Which aspects of the outcome evaluation plan best fit with your organization’s priorities?
       You may have to illustrate your program’s contribution to community priorities to ensure continued support
        If this is the case, then it may be wise to evaluate those aspects most likely to contribute to the community’s interest (assuming that those are also the ones most likely to result in measurable changes)
• Other factors to consider when deciding on an evaluation type
o Obtain community members support
o Establish healthy community committee – partnership with population from all levels should be actively engaged in the program
o Conduct situational assessment to understand the needs and preferences of the people
o Develop a healthy promotion plan
o Implement plan
o Generate evaluation report

Conducting Outcome Evaluation

Questions to Ask and Answer
• How can we use outcome evaluation to assess the effectiveness of our program?
• How do we decide what outcome evaluation methods to use?
• How should we use our evaluation results?
• How can we determine to what degree we have achieved our communication objectives?
• How can we make our communication program more effective?
Conducting Outcome Evaluation: How Well is the Program Working?
• Conduct outcome evaluation by following these steps:
o Determine what information the evaluation must provide that help to answer important questions, such as
     Did the program work? Why? Why not?
     Should we continue the program?
     What can be modified to make the program more effective?
    What evidence proves that funders should continue to spend their money on this program?
o Define the data to be collected
     Knowledge: What people learn about a subject (e.g. the short- and long-term health risks of smoking)
    Attitudes: How people feel toward a subject (e.g. smoking is dangerous to their health)
    Skills: How peoples’ skills and abilities affect a problem by themselves (e.g. variety of ways to say ‘no’ to smoking and awareness of smoking cessation classes)

    Behaviors: How people actually change their way of doing things (e.g. a measurable decrease in participants who smoke)
• Decide on data collection methods
• Develop and pretest data collection instruments
• Collect data
• Process data
• Analyze data to answer the evaluation questions
• Write an evaluation report
• Disseminate the evaluation report

Posted by
Welfare Jambo
Blog
Welfarejambo.blogspot.com