You are here

“Adults really don’t know what they’re doing” and Other Presidential Learnings at ATD 2018

The Wainhouse Research Blog

News & Views
on Unified Communications & Collaboration


“Adults really don’t know what they’re doing” and Other Presidential Learnings at ATD 2018

By: -
12 May 2018

Last week, President Barack Obama keynoted the international learning and talent development tradeshow, ATD 2018, in San Diego California. Most of the 13,000 attendees packed in to hear Obama share insights based on his experience as a candidate and president.

During the discussion with ATD CEO Tony Bingham, Obama addressed resilience. And he, as leader of the free world, related after a long frustrating day trying to work through some complex, contentious issues, he would relate to his school age daughters, “Adults really don’t know what they’re doing. It’s amazing that lots of things work at all.”

While this is funny and true, what underlies the statement is that we (and especially if you are the president) often don’t have as much information or time as we’d like, so that makes coming to a decision or building consensus quite difficult. Obama did share his process of collecting as much information as possible and coming up with contingencies and being open to input.  Be true to your values and facts, and, yes, values and facts do matter.

What’s another thing we adults don’t do so well? It’s measure the business impact of learning programs. Maybe this is L&D’s long day in the situation room. It’s frustrating because the methodology is well known – it’s now 59 years since Dr. Don Kirkpatrick published the series of articles that would become the best-known learning evaluation model, which has Business Results as its highest level. It’s hard work to create a solid evaluation plan, because in addition to good materials and instruction, you need to understand what behaviors need to change, what business results will result from the new behaviors, and how to measure those changes. None of these problems are trivial and all require the cooperation of your business partners.

Kirkpatrick Model case study

Jim and Wendy Kirkpatrick are the principles at Kirkpatrick Partners, a consultancy that trains and consults on updated version of father (and father-in-law) Don’s model. In a role-play exercise in their session, “Overcoming the Challenge of Evaluating Outsourced and Off-the-Shelf Training,” they demonstrated how obtaining the understanding and assistance of your business partners can be a challenge, but not an insurmountable one. Jim summed it up nicely, saying you want more than “sign off,” you need them to “sign on;” otherwise how will you gauge improvement in the specific behaviors and ensure proper support such that they become routine? If you are hesitant about engaging your business partners, why not try a role-play session or two?

Measuring business impact and even ROI is possible. In fact, Jack Phillips of the ROI Institute led a panel titled “Measuring the Impact and ROI of Soft Skills Programs.”  While he did point out soft skills are needed for innovation, sustainability, and creating a productive and healthy work environment, what relates more directly to this blog is the sales training case study Paul Leone of Verizon shared.

Verizon Leadership University developed a soft skills program for new sales managers, which was well received by the learners – 4.7 out of 5 for overall satisfaction (Level 1). Nearly all the learners (97%) gained new and valuable knowledge (Level 2) and 90% were observed applying the new skills on the job. Those trained were able to increase sales significantly, slightly more than twice the cost of the training, resulting in an ROI of 113%. They noted that manager support was a very important factor – those who had strong manager support had four times higher ROI than those who lacked immediate manager support after the training.

And it wasn’t just sales training, the panel also shared two more case studies, an onboarding program at VMWare (93% ROI) and coaching program at Nations Hotel (221% ROI!).

A random walk across the show floor shows that leading learning platform vendors are aware that demonstrating impact continues to be a challenge and are enhancing the tracking and reporting capabilities of their products. Both Docebo and Saba include not only a good set of reports but also have easy-to-use report editors. Abobe’s Captivate Prime LMS includes customizable surveys to collect Level 1 (Learner Satisfaction) and Level 3 (Behavior) data.  And Skillsoft offers usage and value dashboards in both its SumTotal LMS and its Percipio Learning Experience Platform. 

These may be enough for a DIY crowd, but those wanting some hand holding can always consult with the Kirkpatrick Partners or the ROI Institute.  Another option is to contract with Educe, a services firm with experts on the Saba, Cornerstone, and Pathgather solutions. They manage the roll out and management of these learning platforms, and a big part of their practice is helping clients measure the impact of their learning programs.

It does seem that the vendors could be doing more, and I know several are looking into ways to improve measurements and analytics, many relying on artificial intelligence.  One vendor to watch is Fuse Universal. Not only did it just raise $20 million in venture capital, but also it has hired Lori Niles-Hofmann as its CLO. I met with her briefly and can say based on her experience at marketing automation pioneer Eloqua, she undoubtedly will have some influence on how Fuse takes its measurements of learner engagement to the next level.

Measuring the business impact of learning programs isn’t easy and takes a disciplined, systematic approach.  As President Obama pointed out, most things that are worth doing are hard. Get some help if you need to, because it’s worth it.