You might not call it Kirkpatrick Level 2 evaluation but this is crucial stuff and you should be doing it (I’m sure most of you are). Neither is it just for formal training in a classroom type environment – it should be implemented for all learning!
This is the third in a series of posts. The first provided an overview of the Kirkpatrick model of evaluation and the second examined the specifics of Kirkpatrick Level 1 Evaluation. This post will focus on the detail of Level 2.
The keyword that Kirkpatrick uses to describe Level 2 evaluation is ‘Learning’.
Kirkpatrick Level 2 is the foundation for further evaluation activity
I believe that evaluation at level 2 is crucial. Why? Because it is the measurement that you use to determine that learning has actually taken place. This is a key foundation for evaluation at the higher levels – ensuring you have determined that learning has occurred before you examine behavior change at Level 3. Failure to confirm that learning has occurred at this stage has the potential to lead to wrong conclusions later (see the scenario in the box below).
Following a training event you may rightly choose to try and determine the effect of that training in the workplace. Following your study the results may indicate that there has been no change in behavior as a result of the training. You conclude that this is the fault of the local line management for not properly supporting the training that you have delivered.
However, if you didn’t conduct any Level 2 assessment evaluation can you be sure that any learning actually occurred in the first place? Perhaps the failure of the training to have any impact in the workplace is a result of your failure to check that the training was delivered, in a way and to a level, where participants where in the best position possible to implement it on their return to the workplace?
You may still find in the future that learning has not been implemented in the workplace. But, if you have data to prove that participants fully understood the training before they left, you can eliminate any doubts about the training quality and focus on identifying other organisational reasons that may be impacting the problem.
Using assessment to prove and improving training…
The scenario above looked at using level 2 assessment to prove that learning has occurred. Of course, for this to stand scrutiny the assessment has to be valid for the training in question. I believe the key to this is developing your assessment strategies as part of the training design process and in concert with personnel from the key areas of the business who will be utilizing your trained product. Read this post to consider this aspect further – Training Design Informed by Assessment Strategies.
Of course assessment is not just about proving that you have delivered what you said you would. It also plays a key role in identifying how your training could be improved. You can read more about assessment proving and improving training at this post – Formative & Summative Assessment: an explanation.
Kirkpatrick Level 2 – opportunities to exploit:
There are a number of aspects of evaluation at level 2 that you can take advantage of:
- Training deliverers can maintain control as the measurement takes place in the training environment.
- Consideration at the design stage enables effective assessment systems to be ‘built into’ the training.
- Modern technologies can generate efficient ways of collating and analyzing assessment results.
- Straight forward to achieve where specific knowledge and/or practical skills are been assessed.
Kirkpatrick Level 2 – threats to mitigate against:
Whilst there are advantages, there are also some issues that you should be aware of:
- If you have to introduce assessment strategies into legacy courses it will be more time consuming than completing this at the design stage.
- It requires clear analysis to ensure that the assessments are valid for the training in question.
- Specific systems need to be established to ensure that the measurement is consistent both across one particular training event and/or a series of training events where you wish to conduct trend analysis.
- More complex to implement for attitudinal based training.
Linking level 2 data with level 1 results…
Dependent upon your environment there could be benefits in linking your level 2 assessment data with feedback results that you may have collected at level 1. This can be useful if you identify a particular area of concern where your measurements indicate that the level of learning seems to be particularly low.
Linking Kirkpatrick level 1 and 2 data may provide some answers. A low scoring module at Level 2 may have received negative feedback at Level 1, eg “the module was rushed and was all theory based”. This could generate some simple design amendments (i.e. a trial introduction of more practical elements) that you can then monitor to determine the impact on the assessment results.
It’s your moral responsibility to check you’re delivering an effective product…..
So you’re not compelled by the foundation for other evaluation activity or the proving/improving case?! Well, what about your moral obligation? Ok, ‘moral responsibility’ might be a bit strong but surely everyone within the organisation has a responsibility to implement some type of ‘quality check’ to confirm that their part of the process is ‘adding value’ and is not resulting in a ‘faulty product’.
Why should the training department escape this common requirement? Developing relevant assessment strategies for the training that you deliver will enable you to achieve this goal – demonstrating that you’re equipping the people within your organistaion, or the organisation of your client, with the skills, knowledge and attitudes that you have previously committed to delivering.
Kirkpatrick Level 2 Evaluation methods of assessment…
This post hasn’t looked at the different assessment methods and techniques that you can employ to measure the learning that has occurred. Do you have any particular assessment techniques that you find useful for certain types of training? Do you disagree with me that assessment is a crucial activity? It would be great to hear about your thoughts and experiences in the comments below.