Evaluating Programs and Human Performance

One model is Brinkerhoff’s Success Case Method (SCM) this model is focused on finding out what about training or other organizational intervention worked. The SCM model is a 5-step process. First, you focus and plan the success case (SC) study; Identify targeted business goals and impact expectations. Second, construct a visual impact model. This includes explicating the major program goals and listing all impacts/outcomes that are hoped for or are expected to result from the program; survey a large representative sample of all participants in a program to identify high impact and low impact cases. Third, conduct a survey research study to identify the best and the worst cases; analyze the survey data to identify a group of successful participants and a group of unsuccessful participants. Fourth, schedule and conduct in-depth interviews (usually via the telephone for approximately forty-five minutes per interview) with multiple success cases to document the nature and business value of their application of learning and identify the performance factors that supported learning application and obstacles that prevented it. Fifth, write-up and communicate the evaluation findings; document, disseminate and tell the story. This approach can be used for any type of intervention designed to improve performance, including training, performance support systems, information solutions, communities of practice, improved feedback systems, informal and semi-structured learning initiatives and social learning initiatives. I would use this model to evaluate my instruction by assessing skills that will allow students to face a world that is continually changing. They must be able to think critically, to analyze, and to make inferences. I would take an active role in making decisions about the purpose of the assessment and the content that is being assessed.

Another model is Patton’s Utilization-Focused Evaluation (U-FE) the cardinal rule in U-FE is that the utility of an evaluation is to be judged by the degree to which it is used. The evaluator focuses on use from the beginning until the end of the evaluation, and during that time, he or she continually facilitates use and organizational learning or any other process that helps ensure that the evaluation results will continue to be used once the evaluator leaves the organization. U-FE follows several steps. Because U-FE is a participatory evaluation approach, the client and primary users will be actively involved in structuring, conducting, interpreting and using the evaluation and its results. Here are the major steps:

v Conduct a readiness assessment

v Identify the “primary intended users” and develop a working relationship with them

v Conduct a situational analysis

v Identify the “primary intended uses

v Focus the evaluation

v Design the evaluation

v Collect, analyze, and interpret the evaluation data use multiple methods and sources of evidence.

v Continually facilitate evaluation use.

v Conduct a metaevaluation; be accountable, learn and improve

I would use this model by considering how the evaluation might be conducted in ways that increase skills, knowledge, confidence, self-determination, and a sense of ownership among the students participating in the evaluation.

Yes, a return on investment and management of resources should also be considered in evaluations of instructional programs. This means they need to see a dollar value placed on the benefits of the training, not just that students/participants believe the training to be valuable to them or they enjoyed the training overall. They want proof that the training program benefited the organization/students in a positive manner.

Other measures to be considered are district/student satisfaction and quality, cost, time, and if the metric is appropriate and measures what it sets out to measure.

A performance problem in my area is how to organize and arrange a class of 25 – 30 students, when there is barely enough room for 25 students. One way to solve the problem in a non-instructional way is through classroom arrangement and organization to maximize the available space.  Another problem-solving strategy is when students need more space they can go to the library, outside or to a lab to do projects requiring more space. I see an informal learning environment working better to assist with solving the problem because it creates opportunities to help people/students and organizations achieve learning and performance goals. Because learning happens anywhere, any time. The learner is inspired to learn because of an immediate desire to know how to do something or understand a topic.

Rieser, R.A, Dempsey, J V.(2002).  Trends and issues in instructional design and technology.  Upper Saddler River, N.J.  Merrill Prentice Hall, c2002.