So often you’ll see incentive companies push the results of some academic study or some “research” that show cash isn’t a good motivator or that long-term incentives drive more results than short-term incentives or that tall people are 22.6% more likely to be motivated by basketballs than less tall people.
There is an old saw that my boss used to say every time I did my annual budgets – “Figures lie and liars figure.” Meaning, anyone can torture numbers to tell any story you want.
However, in many cases, the research is really a poll some company that sells merchandise conducted, vs. a real scientist-designed test of motivational and behavior change impact.
(#PROTIP: Polls are not studies. Studies may not be research. Private companies selling a product that ask their client what works isn’t valuable.)
But sometimes a study with the appropriate pedigree crosses my twitter feed.
That happened recently.
@katy_milkman – a Wharton Behavioral Science professor tweeted a link to: “Megastudies improve the impact of applied behavioural science”.
It is behind a paywall and right now I’m too cheap to fork over the $30 today to read the whole thing. Check back in the future.
But the abstract gave me enough ammunition for a quick Friday post.
From the Abstract
The study was designed to help uncover what types of applied behavioral interventions provide the greatest behavior influence. Conducted with over 61,000 participants, designed by over 30 scientists, from 15 different universities, the group created 54 different 4-week interventions focused on encouraging exercise at 24 Hour Fitness locations.
Three things popped out of the abstract and the graphic Katy put in her tweet that I think anyone who runs incentives or influence programs for their own employees or their distribution channels should pay attention to.
- 574 forecasters incorrectly predicted what would work the best.
- Only 8% of the interventions induced “significant” behavior change after the 4-week intervention
- DURING the interventions, 45% of the interventions produced 9-27% increases in gym visits.
- Even experts get it wrong. If a huge number of experts can’t predict what will work how can us lowly prols be able to figure this out? Simple. You do a lot of variations on a program/intervention. Just like this study did (but in a rear-view method), having a lot of different program designs gives you the ability to find out what works best. This is why I always advocate for short term incentives, varying the way people earn. This allows you to find out where the real energy is in the program. Test, test, test.
- Only a small number of the interventions had significant momentum outside the test time interval (4-weeks). I think this speaks to the issue of continual communication and engagement with your audience. Keep talking. Keep connecting. You can’t expect anything with a 4-week shelf life to impact someone for any length of time. Sporadic. Different. Variable. All key terms to keep in mind when designing your programs.
- Almost half the interventions produced significant results – up to almost 30% increases! Programs work. But not every program works. That is a key understanding many program designers can’t live with. Remember the words attributed to Abraham Lincoln (though what he actually said was a bit different):
“You can fool all the people some of the time and some of the people all the time, but you cannot fool all the people all the time.”
In other words – you need multiple options/designs/rules/awards in order to cover the “all the time” part of that statement. Your program design needs to be broad and narrow at the same time. How zen, right?
And, while I have not dug into the recesses of the paper – this is for health and fitness behaviors. I don’t know if the results would be different in a different domain – say saving money – or cleaning the house – or selling widgets. So take some of this with a bit of a grain of salt.
Click to Enlarge
My quick look at the graphic included with one of Katy’s tweets showed the following to be the top 10 interventions based on range of average weekly gym visits.
- Bonus for returning visits after a missed workout (don’t punish the miss – reward the return)
- Higher incentives – yup… slap your forehead.
- Social norms – always love this one!
- Free audio book provided – I’m assuming with each visit – nice value-add – almost like getting that first-class upgrade for nothing huh?
- Giving a choice of framing: loss vs. gain-based incentives – I’ll need to read the paper to figure that one out. 😊
- Planning assistance – help me find the path.
- Exercise commitment contract explained – big miss in most programs – ask them for their commitment to come to the gym and lo and behold they are more likely to do that.
- Encouragement of following the path – repeated checkins and showing someone is noticing their behaviors. THIS ONE IS HUGE!
- Asking questions about workouts – see above.
- Fitness questionnaire with Decision Support/Cognitive Reapraisal prompt – not sure what that is but the core is it is a communication element tied to individual situations – in other words – showing participants they are valued and important enough to have someone else paying attention to their progress.
I think if you review the list, you’ll see many of the successful interventions ultimately rest on the fact that someone is paying attention to participants efforts (can anyone say “Hawthorne Effect?). I wonder if that is all this is measuring.
So – have a great weekend. Think about your own incentives and influence strategies. Is there anything on the list that screams out to you as something you want to try?
Let me know and I can comment and post about some of your favorite interventions.