This is the second in a series of posts. The first post provided an overview of the Kirkpatrick model of evaluation. This post will focus on the detail of Level 1.
The keyword that Kirkpatrick uses to describe level 1 evaluation is “Reaction”. Donald & Jim Kirkpatrick (2006)Kirkpatrick, D & Kirkpatrick, J. (2006). Evaluating Training Programs (Third Edition). San Francisco: Berrett-Koehler Publishers, Inc. suggest that evaluating reaction is the same as measuring customer satisfaction. They believe that it is important to understand what people feel about the training they have received.
The suggestion is that people will learn more if they are content with both the environment and the way in which they are learning. This can be particularly important if those individuals will be discussing the training with others who have yet to receive it. Unhappy “customers” spread the word and this could result in the next lot of participants arriving with negative preconceptions before the training even begins.
Or worse. The person who pays for the training hears about the negative responses and decides he isn’t going to fund it anymore!
Criticisms of Kirkpatrick Level 1 Evaluation
Level 1 evaluation receives its fair share of criticisms. The method of collecting this data is frequently referred to as using ‘happy sheets’. This term (which makes me grimace) can be used in a derogatory way. The implication is that you’re only interested in whether people are ‘happy’ with the training, rather than understanding what value is added as a result of the training.
I can accept this argument, if you are only conducting level 1 evaluation. But the point is that you should also be conducting the higher levels of evaluation too, building what Kirkpatrick refers to as a ‘chain of evidence’. This will help you generate a balanced overview of the training. The Kirkpatrick model of evaluation is not intended to enable you to pick and choose which levels you’d like to complete.
Only conducting level 1 evaluation is popular because it is cheap and easy to do. Whilst it’s better then doing nothing, in an ideal world it would be part of a broader plan – check out this post on 5 reasons why you should have a training evaluation strategy.
Kirkpatrick Level 1 – opportunities to exploit:
There are a number of aspects of evaluation at this level that you can take advantage of:
It demonstrates that you care about what people think about the training that you deliver.
You can offer a genuine explanation (in person) of how much you value feedback from those who have received training and that (following sufficient analysis) you are prepared to change the training where appropriate.
You have a captive audience and so should be in a position to generate a 100% return rate.
It’s easy. It is straightforward to design simple techniques to collect this type of evaluation data (I’m developing a ‘how to’ video post to illustrate the design process).
You can easily capture quantitative data but can also generate qualitative information through periodic / end of course discussion groups.
There are no significant costs involved.
Kirkpatrick Level 1 – threats to mitigate against:
Whilst there are many advantages, there are also some issues that you should be cautious of:
It’s easy to just use a standard template rather than to think carefully about the specific questions you may wish to ask. Take the time to tailor your immediate feedback form (I prefer this title to happy sheet) to your particular training.
Don’t make it an afterthought once the training has finished and everyone is itching to get on the road – schedule time in throughout the training.
You can suddenly be faced with lots of data. Make sure you have a system to quickly analyse your returns so you don’t commit the crime of piling the feedback into a cupboard without properly analysing it.
Different people respond to training in different ways. You need to be cautious about ‘knee jerk’ reactions and only look to amend training once you have identified a trend (this may mean considering results over more than one course instance).
Linking Level 1 data to other levels…
There are also opportunities to correlate data you collect at this level with further data that you might collect at level 2 and level 3. More detail on why you might want to link level 1 and 2 results is at this post – Kirkpatrick Level 2 Evaluation.
Heading out to dinner?
If nothing else, consider this…after you have eaten your meal in a restaurant it would be reasonable of you to expect the waiting staff to enquire whether you enjoyed the meal. It is courteous and it is their opportunity to gain some immediate feedback.
If everybody was complaining about the fish then it might be worth the chef taking a closer look. In this respect, I don’t see why delivering training should be any different to delivering food (not the fish bit, I’m sure you get the gist).
Do you employee Kirkpatrick Level 1 Evaluation?
Do you have any success stories using Kirkpatrick Level 1 evaluation? Do you think it is useful or a waste of time? It would be great to hear what you think in the comments section below.