Measuring Employee Learning: How do you do it?
Mr Kirkpatrick and friends
If you play in this world of employee learning – and you too have been trying to figure out the impact it is or isn’t happening – you mostly like know the usual suspects.
The Kirkpatrick four levels of evaluation is the big fish in the pond.
The Success Case method is intriguing.
I’ve even got this book and this book on coaching on my shelf, both definitive tomes…
But, holy cow, it’s difficult
Certainly a few of the my corporate clients have the structure and the discipline to collect data beyond the “smiley sheets” at the end of a session.
But even they struggle to do much with the data. I could be wrong, but it often feels like a defensive position: “If we get challenged, we’ve got this in our back pocket and we can dazzle them with numbers.”
Yet we know from Dan Pink that we’re motivated by purpose, autonomy and mastery. And it doesn’t take much to see how employee learning feeds into all of that.
And we also know that people are busier than ever, and it’s harder and harder to find the time (and sometimes the budget) for employee learning.
So it’s important to be able to make the case that investing in employee learning matters.
Start with questions
You’ve probably experienced a post-event “smiley sheet”. To me they often feel way too many self-serving questions and my answers to them will have no impact on much of anything.
So we’ve got to be rigorous and parsimonious with the questions we ask. And we’ve got to start big.
What are we trying to achieve as a business?
What’s the data people care about?
What’s the impact we want to have at a human level?
How can I connect the data to metrics that others in power might care about?
(What would you add?)
Here’s an inadequate potpourri of answers & approaches
You can tell that this post is more a fumbling grope in the darkness, rather than a neatly packaged answer. I still don’t know how to do this well.
Here are some things I’ve done in the past. Would you share your ideas with me? I’d love to hear what works (or hasn’t worked) in your approach.
Ask simply: would you recommend a colleague take this program? For what reason?
Ask people to write for 3 minutes without stopping telling me whatever you what to say about the program
Ask people to identify 3 things you’ll do differently as a result of this program
Ask: What strategic purpose does this program support, if any? How will it help you achieve that strategic goal, if any? How confident are you about that answer?
Michael, Here’s a few thoughts on this subject of making learning stick or its ROI.
1. Another book for your shelf is “The Transfer of Training” by Mary Broad and John Newstrom.
In fact it should be on your bedside table, not just your book shelf -;) The book sets up a contextual or strategic framework that makes this topic more of a proactive planning process than a reactive ass-covering justification activity.
2. All 8 questions you ask (and in fact all the ROI material) are far more powerful as planning tools than providing feedback data. As you say, by then it’s too late.
3. The L&D terms, “competency” and “capability”, need to be defined as “anything an employee does well that drives meaningful business results.” We need to look further than (EG) test/ pop quiz results to show that something is sticking.
In summary, this issue is summed up by the saying, “what I focus on I’m more likely to achieve.”
Mark
Mark – great stuff. Appreciate the book recommendation and I agree with your other points.
I would ask a couple of the same questions 2 months after the program, in a slightly different way. New habits or learning would be hardwired with visible impact by then.
—-
Ask: Would you still recommend a colleague take this program? For what reason?
Ask: Identify 3 things you have done differently as a result of this program. (May be useful reflection question for many participants.)
—-
On observing myself and others, most of us get busy quickly with the requirements of daily work and may not even find the time to respond to a past training unless it made a difference. Even a high response rate may indicate how valuable the program was to the participants – they took time and cared enough to respond.