Training-Program Evaluation Design and My Experience in Japan

No one training-program evaluation design is all inclusive or overarching; several factors need to be examined very seriously in determining which design is the most suitable (Hollenbeck & Noe et al, 2010):

• The size of the training program
• The purpose of the training
• The implications, or ramifications, if a particular training approach does not work
• Company norms relative to evaluations
• The costs of evaluation design and implementation
• The amount of speed needed for procuring training-program effectiveness information

Additionally, several questions need to be asked to determine which evaluation design is the most appropriate (Noe, 2010):

• Is the training program under consideration modifiable?
• Would ineffective training adversely impact customer service, product development, safety, and/or interpersonal relationships between and among employees?
• How many employees are being considered for training?
• Is the training intended for learning only, results only, or a combination of both?
• Is exhibiting the results a part of the organization’s culture?
• Can a complicated study be readily analyzed and disseminated?
• Is the evaluation cost ineffective?
• When is the information required?

In the case of the English conversation school chain that I worked for in Japan (1989-2005), a pre-test/ post-test comparison design would have been necessary to measure the efficacy of teacher training. Teachers prior to training should have been observed and then trained, observed, and evaluated at the one month, three month, six month, and one-year mark. Such a construct would have been the most even-handed manner to train the instructors and to measure their performance and the efficacy of the training on a consistent basis.

The training actually given from 1992 onward was merely feedback given to instructors by trainers and head teachers in the aftermath of evaluation at the halfway and ending points of each one-year contract. No consideration or effort was made by the company to design and implement practical hands-on training for the teachers or for the trainers and head teachers, which, over time, led to a significant decline in teaching morale and performance and an attendant slide in student satisfaction with the company’s schools.

Sources: Hollenbeck, John R. & Noe, Raymond E et al. (2010). Human Resource Management: Gaining a Competitive Advantage, pg. 318, 7th Edition, McGraw-Hill Irving, New York, NY.

Noe, Raymond E. (2010). Employee Training and Development, p 239, 5th Edition. McGraw-Hill Irving, New York, NY.