Logic Model Component:
Measures the extent to which intended users apply knowledge to improve practice guidelines, program design and management, training curricula, or research practice
This indicator measures the extent of the use, and the outcomes of the use, of knowledge gained from KM outputs to improve practice guidelines, program design and management, training curricula, or research practice, resulting in better service delivery, more efficient programs, better training and education of health care personnel, or stronger research designs.
Description of knowledge from KM outputs that was used, approximate timeframe of use, organization(s) involved, how programs or practice benefited from applying the information, and any further outcomes associated with use
To obtain a quantitative data, evaluators can count the instances of use of knowledge gained from a KM product or group of products. Alternatively, evaluators can calculate the percentage of respondents to a survey who said that they used knowledge gained from the KM product.
User surveys (online, mail, telephone), usually distributed after the product has been disseminated; informal (unsolicited) feedback; in-depth interviews (telephone or in-person); guidelines or protocols referencing or incorporating information/knowledge from KM outputs
Please indicate whether or not you have used information from the [Web product] for the following purposes. (Select all that apply.)
o To make management decisions (either personal or organizational)
o To design or improve projects or programs
o To develop or improve policy or national service delivery guidelines
o To develop training programs or workshops
o To assist in designing education materials
o To guide research agenda or methods
o To put research findings into practice
o To promote best practices
o To write reports/articles
o To develop proposals
o To increase public awareness
o To increase my own knowledge
o Other, please specify __________
Please give an example of how you have used specific information from the [Web product] in your work. (Open-ended.)
Please rate the following statements about performance areas affected as a result of using the [Web product]:
(1-Strongly disagree, 2- Disagree, 3-Not sure, 4-Agree, 5-Strongly agree)
o Based on something I have learned in it, I have changed the way I perform my job.
o I have used information from it to improve my skills.
o It has helped me to be more competent and effective at my job.
o It has helped me to perform my job more efficiently.
o It has helped to improve the performance of my organization.
Please give a specific example of how the [Web product] has improved your own performance or your organization’s performance. (Open-ended.)
In the 2011 K4Health website users’ online survey, majorities of respondents (n=224) used the information obtained from the K4Health website to improve their knowledge (72%), to design or improve projects or programs (55%), and to promote best practices (52%).
In the survey about the LeaderNet webinar on blended learning, when asked for examples of how they applied or plan to apply their new knowledge to their work, participants stated they will apply the ADDIE model (consisting of 5 phases—analysis, design, development, implementation, and evaluation), set SMART objectives (consisting of 5 criteria—specific, measurable, attainable, relevant, and time-bound), thoroughly analyze the target audience, measure learning interventions beyond Kirkpatrick’s Level 1 and 2 (reaction and learning), apply blended learning strategies to their current learning challenges, and engage in Global Health eLearning courses.