Menu
Advanced search Subscribe
Resources

Home > Resources > Getting Started Guides > Research evaluation

Introduction to...

Evaluation


Definitions

Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products and organizations to improve their effectiveness.
(American Evaluation Association, 2002)

Evaluation is the process of obtaining and disseminating information of use in describing or understanding the particular program, or making judgements and decisions relating to past, existing or potential programs.
(Australasian Evaluation Society, 2002)

Effective evaluation is not an 'event' that occurs at the end of a project, but is an ongoing process which helps decision makers better understand the project; how it is impacting participants, partner agencies and the community; and how it is being influenced/impacted by both internal and external factors.
(W.K. Kellog Foundation Evaluation Handbook, 1998)

Purposes of evaluation

1. Evaluation as an audit tool for accountability: measurement of results, appropriateness, efficiency.

2. Evaluation as a tool for development: program improvement, capacity building, network creation, social cohesion, and social capital.

3. Evaluation for knowledge creation: developing a better understanding in some area. Developing local knowledge that is needed for local development, and possible transfer of local knowledge.
Chelimsky E. (1997). Thoughts for a new evaluation society. Evaluation, 3(1), 97–109

Guidelines for conducting evaluations

Program Evaluation Standards (developed by the American Evaluation Society):

Utility — The utility standards are intended to ensure that an evaluation will serve the information needs of intended users.

Feasibility — The feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal.

Propriety — The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results.

Accuracy — The accuracy standards are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth or merit of the program being evaluated.

Guiding Principles for Evaluators
The Australasian Evaluation Society Ethics Committee has developed Guidelines for the Ethical Conduct of Evaluations. The Guidelines refer to three main stages of program evaluation: commissioning and preparing, conducting, and reporting. They can be downloaded in a PDF format, or printed copies can be purchased.

Checklists, tools and guides

Guidelines for the Ethical Conduct of Evaluations
Australian Evaluation Society: These guidelines are widely used by organisations as a training resource and to inform their evaluation policies. They are intended to promote the ethical practice of evaluation, stimulate discussion, and to help resolve ethical issues that arise in the course of an evaluation.

Resources for Methods in Evaluation and Social Research
A website that links to free on-line books, manuals, and articles on how to do evaluation and social research, including surveys, interviews, observations and more. The site also links to research about research methods, free software, and other sites on how to present data.

The Centers for Disease Control and Prevention (CDC): AFramework for Program Evaluation
This website provides a framework to inform public health program evaluation. It also provides access to a range of evaluation resources including an Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide [PDF 2.8MB], on-line tutorials, tools, newsletters and handbooks.

Evaluation Checklists
A website created by The Evaluation Center at Western Michigan University which aims to improve the quality and consistency of evaluations. It includes a range of checklists addressing specific evaluation tasks and approaches.

InnoNet.org Resource Center
This page of the Innovation Network website provides a wide variety of information and website links, from definitions to tools, all designed for both general inquiries and those seeking a more in-depth study.

Program Planning Evaluation Wizard
An Australian software tool designed to assist project officers working on primary health care and health promotion projects. It is a step-by-step guide on how to: develop a case for projects; develop project plans and evaluation plans and; write project reports.

Research Knowledge Base
Comprehensive web-based textbook that includes simple coverage of sampling, measurement, design, analysis, interviewing and survey research.

Training & support

AREOL is "action research and evaluation on-line", a 15-week introductory course offered by email twice each year as a public service by ALARA, the Action Learning Action Research Association Inc.

RMIT University offers Short Courses and Single Courses to health professionals and practitioners, across seven faculties, with a number relevant to evaluation.

Evaluation associations

Australasian Evaluation Society (AES)
This a professional organisation for people involved in evaluation. The AES has over 1 000 members involved in all aspects of evaluation and performance measurement. Members include evaluation practitioners, managers, teachers and students of evaluation, and other interested individuals.

International evaluation associations:

Useful resources

 Electronic Discussion Lists
The American Evaluation Association website lists a large number of links to electronic discussion lists. These are topical, email-based discussion groups devoted to evaluation issues.

 Building skills and knowledge in program evaluation [PDF 212KB]
Presentation given by A/Prof Libby Kalucy

 Practical Review and Evaluation—Valuing primary care [PDF 65.4KB]
Presentation given by A/Prof Libby Kalucy

 Divisions, evaluation and the new performance framework [PDF 101KB]
Presentation given by A/Prof Libby Kalucy

 Evaluation in primary health care [PDF 118KB]
Presentation given by Anne Magarey

 What is evaluation? [PDF 158KB]
A brief and simple fact sheet about evaluation compiled by the University of NSW (March 2002)

 Advocacy Evaluation Update
The Center for Evaluation Innovation offers original articles and updates on advocacy evaluation theory, methods, and practice and is for people who are working to influence decision makers, and looking for a way to measure their success.

 Transforming Evaluation
The Innovation Network has resources and news geared toward helping organisations to engage in meaningful evaluation.

Books

Evaluating Health Promotion Programs
Ontario Agency for Health Protection and Promotion (Public Health Ontario), Snelling S, Meserve A. (2016). Evaluating health promotion programs: introductory workbook. Toronto, ON: Queen’s Printer for Ontario. [PDF 1.9MB]
The reference list alone is worth accessing: a long list of resources that address types of evaluation, qualitative and quantitative methods, data collection, questionnaire design, etc.

Sage Publications
Sage Publications have a number of book series relevant to evaluation, such as the Applied Social Research Methods series and a Program Evaluation Kit series (includes Evaluator's Handbook, How to Design a Program Evaluation, How to Assess program Implementation, How to Communicate Evaluation Findings).

Journals

Evaluation Journal of Australasia
…the journal of the Australasian Evaluation Society (AES), published twice a year, to further the aims of the AES to improve the theory, practice and use of evaluation.

Evaluation Review
…provides a forum for researchers, planners and policymakers engaged in the development, implementation and utilization of evaluation studies. Reflects a wide range of methodological and conceptual approaches to evaluation and its many applications.

Evaluation: The International Journal of Theory, Research and Practice
…publishes original evaluation research, both theoretical and empirical, as well as reviews of relevant literature and overviews of developments in evaluation policy and practice.

BMJ Quality and Safety
…in order to address the issue of quality, this journal publishes articles relevant to the theories and practice of evaluation. It is an interdisciplinary journal published bi-monthly.

Feedback

Was this guide useful? We welcome your comments and suggestions, please use the feedback form and let us know what you think.

Compiled by PHCRIS
Last updated Tue 31 Oct 2017
Suggested citation
Primary Health Care Research & Information Service (2017). PHCRIS Getting Started Guides: Introduction to... Evaluation. From http://www.phcris.org.au/guides/evaluation_gettingstarted.php (Accessed 11 Dec 2017)