Should we be sulking about Kirkpatrick?

I have had no issue with the Kirkpatrick model, as it has always made sense to me. Just recently I have been challenged to reconsider my position. Last week running the Learning Loop, one of the participants said they have started to use Will Thalheimers’ Learning Transfer Evaluation Model, which made me curious about what that model had to offer, that was different to Kirkpatricks.

This week I read a blog from Work Learning Research  saying how levels 3 and 4 of Kirkpatrick are misunderstood by most L&D people, that they use learner reactions as if they were valid level 3 and 4 measures. The blog got me ruffled a bit, particularly because the survey results:

1) use a sample size of TWO for vendors

2) there is no visible link with those 250 surveyed and whether they are actually even measuring at levels 3 or 4 (if they are not, then could they know the answer to the question?)

This makes any conclusions, drawn from the research, in my opinion tenuous to say the least.

What made me curious enough to look at Will Thalheimers’ model was any suggestion that it had something new to offer. What I did like was the focus on learning transfer, but then I thought – shouldn’t the focus be on performance? Will has a lot to say and criticise the Kirkpatrick model for, but I am looking for a deeper and better way to do evaluation. What I would like to offer is a deeper explanation of Kirkpatrick’s model and I hope you will make up your own mind how you might apply it.

“Begin with the end in mind” said Steven Covey. I agree wholeheartedly. If you do a thorough analysis, engage with stakeholders and line managers about the outcomes (in performance) required, then evaluation should be straight-forward. But please note if the analysis part is skipped over or done on a superficial level, THE EVALUATION WILL BE DIFFICULT WHICHEVER MODEL YOU USE.

  1. Level 1 – learner reactions – still an important part of seeing if you have had sufficient engagement and struck the right chord with the objectives (Business focused and learner-centred according to the 5 Secrets of Accelerated Learning.
  2. Level 2 – learning achieved – did they learn what they were supposed to and have they met those objectives? This is important to know for L&D and their line managers
  3. Level 3 – impact on performance – this is what is then observed on the job and in my opinion should not be L&D’s sole remit. If they have analysed the needs correctly and determined the correct outcomes, the line managers should be engaged enough to help imbed the learning as well as measure performance improvements.
  4. Level 4 – impact on the business- if the stakeholders have been engaged and the outcomes are focused on the business, the stakeholders will be interested in measuring the business impact.

Here are some ways in which you could do evaluate at different levels of Kirkpatrick. Let me know what you  think – I will be curious to hear your reactions!

In the graphic below I have overlaid Kirkpatrick model onto Will Thalheimers’ model.

Evaluation methods answers

 

 

2 Comments
  1. Andrew Hurren 3 months ago

    Sorry but I’m really not sure what you are getting at with this article. The additional steps in Thalheimers model seem largely implicit in Kirkpatrick, if you have studied it properly. I’m also concerned that you completely ignore the new “pre step” in the Kirkpatrick process, the consultation. Establishing what the desired outcome is for the business. Identifying this and establishing meaningful KPIs, to indicate what good looks like, is critical to levels 3 and 4.
    Kirkpatrick isn’t just an evaluation process, it’s a continuous improvement tool. People don’t misunderstand the first two levels. They stop there because getting the meaningful data from the 3rd and 4th level is challenging and pretty much impossible if you haven’t done a good job at the consultation stage.

    • Author
      Krystyna Gadd 3 months ago

      Hi Andrew
      The point of this article was summed up in the paragraph before the Kirkpatrick picture and agrees with your points:

      “Begin with the end in mind” said Steven Covey. I agree wholeheartedly. If you do a thorough analysis, engage with stakeholders and line managers about the outcomes (in performance) required, then evaluation should be straight-forward. But please note if the analysis part is skipped over or done on a superficial level, THE EVALUATION WILL BE DIFFICULT WHICHEVER MODEL YOU USE.”

Leave a reply

CONTACT US

Please leave your message here and we will get back to you as soon as possible.

Sending

©2018 Learning Professional Network

Privacy Policy