Colin Braman’s Program Evaluation Blog

In every human society, regardless of how it is organized, individuals must figure out how to meet their needs with limited resources (Ayittey, 2018). In most modern societies, these needs are met primarily in the marketplace; we voluntarily offer our goods and services in exchange for those of others. Whether due disease, disability, natural disaster, or imperfect social institutions, some people do not have their needs met in the market. Governments and non-profit organizations respond by creating social programs—funded by charitable contributions or taxation—to offer services to improve the lives of the less fortunate (Newcomer et al., 2015).

Social programs encompass a wide range of activities, from government financial assistance programs (such as SNAP) to community arts and cultural enrichment programs. Although these government and non-profit programs are designed with the best of intentions, experience shows us that social problems are difficult to alleviate (Rossi et al., 2019). We also run into the “economic problem” of living in a world with scarce resources and competing needs. Resources spent pursuing ineffective strategies are resources not spent on better options. As Thomas Sowell (2015) reminds us, “…there are no ‘solutions’ but only trade-offs” (p. 586).

In a world of competing needs and scarce resources, program staff, managers, and policy makers must have credible evidence to identify what interventions are effective in serving their communities and how these interventions can be improved. Program evaluators use social science research methods to provide evidence of whether a program is effective and how it can be improved (Patton & Campbell-Patton, 2022).

Michael Scriven (1991) defines “evaluation” in its most fundamental form as “…the process of determining the merit, worth, or value of something…” (p. 139). Program and policy evaluators specialize in determining the merit, worth, and value of social interventions, but their methods may be used in applied research projects of all kinds. A definition of program and policy evaluation that I like is offered by Carol Weiss (1998), who describes it as “…the systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program or policy” (p. 4).

Program evaluators—with the help of program stakeholders—use quantitative and quantitative research methods to answer questions such as:

• Are staff implementing the program as intended?

• Are the people served satisfied with the program?

• Is the program producing the desired outcomes?

• Does the program produce better outcomes than an alternative program?

• What are the strengths and weaknesses of the program? How can it be improved? (Alkin & Vo, 2018; Weiss, 1998).

The purpose of this blog is to describe and discuss different evaluation methods and theories for the benefit students, practitioners, and program staff and managers. It will also help me to continue expanding my knowledge of new developments in the field.

References

Marvin Alkin, & Anne Vo (2018). Evaluation Essentials: From A to Z (2nd ed.). The Guilford Press.

George Ayittey (2018). Applied Economics for Africa. Atlas Network. Retrieved from: https://www.africanliberty.org/appliedeconomics/ .

Kathryn Newcomer, Harry Hatry, & Joseph Wholey (2015). Handbook of Practical Program Evaluation (4th ed.). Jossey-Bass.

Michael Quinn Patton & Charmagne Campbell-Patton, C. (2022). Utilization-Focused Evaluation (5th ed.). SAGE Publications.

Peter Rossi, Mark Lipsey, & Gary Henry (2019). Evaluation: A Systematic Approach (8th ed.). SAGE Publications.

Michael Scriven (1991). Evaluation Thesaurus (4th ed.). SAGE Publications.

Thomas Sowell (2015). Basic Economics: A Common Sense Guide to the Economy (5th ed.). Basic Books.

Carol Weiss (1998). Evaluation (2nd ed.). Prentice Hall.