Number/percentage of intended users applying knowledge gained from a KM output to inform policy

Indicator Number: 
42

Category: 

Logic Model Component: 

Data Type(s): 
Count, proportion, qualitative
Short Definition: 

Measures the extent to which intended users apply knowledge either to change or enhance existing policies or to develop new policies at any level of the health system

Definition and Explanation (Long): 

This indicator measures the use of knowledge gained from KM outputs in policy formulation and the outcomes of that use. It covers efforts either to change or enhance existing policies or to develop new policies—at any level of the health system. Policies both reflect and affect the public interest and are considered keystones or necessary tools in making public health improvements.

Data Requirements: 

Self-reported information from audiences using the knowledge to inform policy; description of knowledge from a KM output used, approximate time frame of use, organization(s) involved, how policy formulation benefited from applying the knowledge, and any further outcomes associated with applying the knowledge

Data Sources: 

Audience surveys (online, mail, telephone), usually distributed after the product has been disseminated; informal (unsolicited) feedback; in-depth interviews (telephone or in-person); copies of policies referencing, incorporating, or shaped by information/knowledge from KM outputs

Frequency of Data Collection: 
Annually
Purpose: 
Like the previous indicator on practice (indicator 41), the number of instances of use of knowledge gained from a KM product or group of products to inform policy can provide a quantitative assessment. Alternatively, evaluators can calculate the percentage of respondents to a survey who said that they used the knowledge gained from the KM product to shape policy.
Issues and Challenges: 
For more insight, it is important to follow up with an open-ended request for specifics. Evaluators can then create a case-study summary of the collected anecdotal evidence. Methodological challenges involved in measuring the role of knowledge in policy formulation include the often competing or reinforcing influences of other external forces or conditions, appropriate attribution, the long timeframe needed for changes to occur, shifting strategies and milestones, and policy-maker capacity and engagement (Reisman et al., 2007). It may not be easy for respondents to recall which particular knowledge gained from which specific KM output was used and how it contributed to the policy.
Sample Topics and Questions for Data Collection Instruments: 

Please indicate whether or not you have used information from the [Web product] for the following purposes. (Select all that apply.)
o To make management decisions (either personal or organizational)
o To design or improve projects or programs
o To develop or improve policy or national service delivery guidelines
o To develop training programs or workshops
o To assist in designing education materials
o To guide research agenda or methods
o To put research findings into practice
o To promote best practices
o To write reports/articles
o To develop proposals
o To increase public awareness
o To increase my own knowledge
o Other, please specify __________

Please give an example of how you have used specific information from the [Web product] in your work. (Open-ended.)

Please rate the following statements about performance areas affected as a result of using the [Web product]:
(1-Strongly disagree, 2- Disagree, 3-Not sure, 4-Agree, 5-Strongly agree)
o Based on something I have learned in it, I have changed the way I perform my job.
o I have used information from it to improve my skills.
o It has helped me to be more competent and effective at my job.
o It has helped me to perform my job more efficiently.
o It has helped to improve the performance of my organization.

Please give a specific example of how the [Web product] has improved your own performance or your organization’s performance. (Open-ended.)

Pages in the Guide: 
60

Published Year: 

  • 2013
Last Updated Date: 
Wednesday, September 6, 2017