2 levels and 2 models for measuring L&D results
Measuring equals knowing. But how to calculate the ROI of Learning & Development? And how to demonstrate that your learning budget is spent responsibly? It’s a challenge for many HR and L&D departments but, according to Springest, not an insurmountable one.
Below, we share a few options to arrive at a good ROI calculation for your investments in learning – or at least a clear business case.
Why the regular ROI model is hard to apply to learning
Return on investment (ROI) = (Expected) return / Investment (costs) × 100%. But what does this look like for Learning & Development?
In most cases, the costs of further training or retraining are not that difficult to calculate. It includes the net investment in training and courses, investments in the development of learning programs, tools and platforms and the costs of facilitating learning. Usually, even the costs of lost productive working hours can be calculate quite easily.
But what exactly is the (expected) result of training and development in concrete amounts? How much more will your business earn simply because an employee is better trained? And how to tell if those extra earnings would not have been there otherwise?
“If learning has value, show the value. Otherwise it will be costs and costs will be cut.”
– ROI Institute
Measuring the result of learning in an organisation
Although the difference between “learning” and “not learning” is difficult to measure, it is less complicated to monitor the learning curve of individual employees. You can keep track of the extent to which they improve, move on to new positions or even leave the company when the time is right from them to take the next step.
You can measure this qualitatively with evaluation forms and appraisal interviews but result measurements such as productivity, customer satisfaction and turnover can also be taken into account. Therefore, what you need are clear goals for each learning path – and an organised system to document learning input (such as the monetary investment and the chosen learning products) and learning output (such as evaluations and next steps).
The output of learning
Training is only successful when the learning goal has been achieved. This can be to apply a new skill or simply to gain more energy, concentration or motivation.
What an employee can do after a training is not the main thing. It is all about what he or she then does with it in the workplace.
The learning goals for organisation are almost always certain desired behaviours. For example, if an employee in the industry learns how to use a certain machine, the desired behaviour is that he or she operates it safely and correctly. If a customer service employee learns to deal with resistance, this should lead to better communication with (dissatisfied) customers and higher customer satisfaction.
In case of compulsory learning due to legislation or regulations, certification will also be an important objective. But passing a test or an exam mainly says something about the retention of the subject matter. It doesn’t say anything about the effect on their productivity, safety in the workplace and/or the operating results.
What has been learned, therefore only has added value if it is actually applied during work. What an employee can do after a training is not the main thing. It is all about what he or she then does with it in the workplace.
You can therefore measure the efficiency of training at two levels:
- Can the employee show the desired behaviour after the training/course/learning activity?
- Does this new behaviour bring the organisation the desired result?
To measure this, Professor Donald Kirkpatrick developed a training evaluation model in the 1960s. His “Four Levels of Learning Evaluation” became the most popular form of training evaluation in the 1970s and 1980s. It is still widely used.
Instead of the ROI, however, Kirkpatrick focuses on the ROE; the Return on Expectation. Jack Philips, founder of the International ROI Methodology, then further expanded this model to include the ROI. We summarise both models below.
Don Kirkpatrick’s training evaluation model
Don Kirkpatrick’s learning evaluation model provides organisations with an effective four-step strategy for evaluating learning activities. The four levels of this model are (1) response, (2) learning, (3) behaviour and (4) results.
These levels work as follows:
Level 1: Response
What is/was the response of the participants to the learning activity? This allows you to gauge whether the conditions for learning were met.
Level 2: Learning
At level 2 you then gauge what the participants have learned from the training. You can use a short quiz or practical test for this.
Level 3: Behaviour
In the third phase, you try to test whether the students have also put what they have learned into practice using various assessment methods.
Level 4: Results
Finally, you evaluate whether the training met the expectations of the participants. In this way you determine the return on expectations: the ROE (Return on Expectations).
With the Kirkpatrick model you gain insight into the value of training activities, but the ROI has not yet been calculated. Jack Philip therefore added a fifth evaluation level.
Jack Philips’ ROI model for learning
The pyramid above was developed by Jack Philips. It consists of two parts: the ROI of learning and the ROI of performance.
The ROI of learning is the literal learning return. As with levels 2 and 3 of the Kirkpatrick model, you measure what the participants have actually learned here. The ROI of performance then shows how that contributed in concrete terms to the operating results and the organisational goals. In this part of the model, you provide insight into the monetary value of L&D.
5 levels of measuring impact
Philips distinguishes 5 levels to map the ROI of learning and performance: (1) reaction/satisfaction, (2) learning/knowledge and skills learned, (3) application, (4) impact/results and (5) ROI.
Level 1: Learning satisfaction
As with level 1 in Kirkpatrick’s model, this section looks at participants’ response to the program, trainer, location, and so on. The most common way to measure this is with an (online) questionnaire after the training. With the right tools and systems, such as our customised learning platform, Springest Go, you can fully automate this process and keep track of all evaluations per learning product and department.
Level 2: New knowledge and skills
At the next level you then measure how the training contributed to the acquisition of knowledge and skills or the uptake of the desired behaviour. Possibilities to measure this include exams or knowledge tests and (online) simulations.
Level 3: Application
At level 3, we arrive at testing the actual application or use of the new knowledge and skills in practice. This is also called transfer to practice. Having a difficult conversation in a safe training environment remains something different than doing the same thing in the “real world”. At this level, we look at the impact on performance. But how do you measure that?
Well-known examples of measurement methods when it comes to performance are the 180/360 degree measurement and observation.
With the 180 degree measurement, you provide a second perspective, in addition to that of the employee himself: the assessment of the manager. In a 360 degree measurement you also ask other colleagues for feedback on what’s been learned. In the ideal scenario, a baseline measurement and a final measurement will take place to properly demonstrate the difference.
With observation, the employee is consciously assessed on the particular behaviour in a suitable situation in the workplace. This could be, for example, listening in, in the case of telephone customer service, or observing a meeting or sales call.
Level 4: Results
At the fourth level you assess whether the goal of the training has been achieved. It is important to ensure that these goals are formulated in such a way that they clearly contribute to business goals.
The goal is therefore not only that the new competencies have been trained, or that participants must rate the program with at least an 8, but also something like a 15% reduction in absenteeism in a certain period, a turnover increase of 20%, or an increase in the average customer rating from 7.5 to 8.
In this case, it is important to perform a zero measurement prior to a certain training program, an intermediate measurement at the end of the program and a final measurement one year after the program, to also determine the effects on the medium or long term.
Crucial and most complicated here is “isolating the effects”. Because to what extent can the improvements really be attributed to the training program?
In the case of an objective such as less absenteeism or more turnover, there are many variables. However, the ROI Methodology offers several techniques to isolate the effect of a training, such as the use of a control group, consulting and comparing trend lines, and the estimates of the participants themselves and/or their supervisors.
Level 5: ROI – Return on Investment
Only when the results have been validly isolated you can map the yield of a training program. For example, has the absenteeism decreased from 3.5% to 2.5% and is this for 35% attributable to the program? Then you can calculate relatively easily what the program has yielded financially by calculating the costs of absenteeism.
It’s important not to skip any items for costs to arrive at a correct calculation. This includes location costs, costs for the trainers, material costs, missed working time of the participants and project costs.
Are the yields and costs clear? Then the last step is to apply the well-known ROI formula: net revenue divided by total costs, times 100.
Documenting learning input and learning output
For both models and for measuring ROE as well as ROI, it is essential to correctly document learning input and learning output. And the only way to do this properly company-wide is by having this neatly arranged at an individual level – preferably all in a single system.
This way you can also make interim adjustments when a business goal changes, department objectives are adjusted, or the central focus throughout the organisation changes.With Springest Go and Springest Certified, Springest helps organisations to gain real-time insight into the first levels of both models. We make it easy to centrally organise each part of the learning process. This applies to training procurement, quality evaluations, making curricula, keeping track of learning objectives and viewing the results or diplomas achieved. And that results in savings.
Stork, for example, says it saves 15 percent on all costs related to learning with our customised learning platform. Arjan de Bruijn, HR Manager at Stork: “Our learning platform makes reports more accurate. We now know what we spend per business unit, manager and employee. As a result, we are no longer dependent on supplier reports, which is of course undesirable. The improved reports also allow us to analyse what we spend at micro level. For example, we now know what we spend on leadership training and can quickly determine when an in-company training is better, instead of three open registrations. ”
Want to learn more? Discover how we help other organisations learn with our tailor-made learning platforms.
