| ROI
Return on Investment or Return on Illusion?
By Kevin J. Fleming, Ph.D.
A CEO calls in his chief engineer and asks
him, “Can you tell me what two plus two is?” The
engineer promptly replies, “It is exactly four.”
Then the CEO has the corporate legal counsel come in and the
CEO asks him: “Can you tell me what two plus two is?”
The lawyer replies, “Well, it is somewhere between three
and five.” Then the CEO calls in the corporate accountant
and asks him, “Can you tell me what two plus two is?”
The accountant goes over to the window, draws
down the shade and says, “What do you want it to be?”
When it comes to assessing ROI in the complex
realm of training and development initiatives, the complexity
of human nature rears its ugly head. And just when you thought
the state of affairs was challenging enough, our own past
comes back to haunt us–those days when we all fell asleep
in statistics classes thinking, “This stuff doesn’t
matter.”
Well, I am here to tell you that it does
matter. You can’t have serious numbers without a serious
science of numbers as well.
When it comes to evaluating my interventions,
I offer serious science as a way of validating and measuring
my work. For instance, I helped a client see that their pre-/post-methodology
of assessing change after a communications workshop was flawed
statistically and therefore in the bottom line analysis financially.
How? Well, imagine if you were a participant at “time
one” having to assess your current level of empathy
with your current workers on a scale from one to 10. And say
you felt you were an average “seven”—not
great, but not horrible either; you then go through your training
and development work withyour consultant and are ready to
take your post-test at “time two.”
At this point, months later, you rate yourself
a seven—not because you didn’t think anything
changed, but because you realized you were really a “two”
at “time one.” This phenomenon is called a “response
shift bias” and is a silent killer to ROI calculations
using the most common change method out there—pre- and
post-assessments. The fix? Measure “pre” and “post”
together at “time two.”
And since the best consulting comes from
asking the right questions, not giving the right answers,
here are some necessary questions to ask in your company when
measuring the effectiveness of your consultants: What is the
organization’s true position on training—a cost
or an investment? Do the decision makers really need to have
training evaluated by traditional ROI models? Understand the
“break even” point. This is not a particularly
sophisticated analytical technique but the lesson is clear.
When is training not worth it? And most important, ask what
matters most: the calculation or getting clear on who is accountable?
Many times, building accountability and taking it seriously
amongst your leaders fixes things more then any razzle-dazzle.
Dr.
Fleming is a regular columnist on Transformation Insights
for the high end executive magazine, Executive Decision.
|
