The Stepathlon is a 100-day international event based around participation of employees in a workplace-based pedometer program. The Stepathlon was conducted annually, with employee participants organized into teams of 5 individuals, to provide a supportive social environment to facilitate activity.
So, you inveigle workplaces to sign up for the Stepathlon program. Then those workplaces inveigle workers to organize into 5 person teams (Uncle Norm, Ms. Comparison). The workers wear pedometers and also hit the Stepathlon app or website to self report on steps, other physical activities, and weight over the course of 100 days.
The people running Stepathlon are a high energy bunch because they have been doing this program for several years now and across the globe. Consider this.
Participants were recruited from 64 countries, from 481 employers, in 1481 cities, on all continents.
That’s a lot of Other Guys and all organized through apps ‘n iGizmos. Virtually no face to face contact with the panthers running Stepathlon and any Other Guys. Let’s count the steps from a 3 year evaluation of the program.
Results: N=69,219 subjects participated (481 employers, 1481 cities, 64 countries, all populated continents, age 36±9 years, 23.9% female, 8.0% high-income countries, 92.0% lower-middle income countries). After Stepathlon completion, participants recorded improved step count (+3519 steps/day 95%CI: 3484-3553, P<0.0001), exercise days (+0.89 days: 95% CI 0.87-0.92, P<0.0001), sitting duration (-0.74 hrs. 95%CI: -0.78 to -0.71, P <0.0001) and weight (-1.45kg, 95%CI:-1.53 to -1.38, P <0.0001). Improvements occurred in women and men, in all geographic regions, and in both high and lower-middle income countries, and reproduced in 2012, 2013, 2014 cohorts. Predictors of weight loss included step increase, sitting duration decrease, and increase in exercise days. (all P <0.0001).
Of course, with nearly 70,000 cases you could detect statistically significant changes in the amount of lint stuck in computer keyboards, so those p values don’t count the change that really counts. Let’s focus on the most useful count: Weight loss. Remember:
. . . weight (-1.45kg, 95%CI:-1.53 to -1.38, P <0.0001)
Across the 3 different years of the Stepathlon, Other Guys self reported losing on average 1.45 kilos, about 3.2 pounds, over the course of 100 days of the program, about a pound a month. Maybe.
The method here is very shaky and does not inspire much confidence in Professor Poopypants. Even before we get real Poopy, realize that we’re dealing with motivated volunteers in the workplace. This is called the self selection effect and arises when you don’t randomly select or assign Other Guys to controlled conditions. People who are already leaning in, if only because their boss is leaning in, are doing this. Think about drawing a random sample of 500 businesses worldwide, inviting all employees to participate, and see what you get. Something like this would be great.
Over the 3 years from 2012-2014, 69,219 participants completed the pre-event questionnaire, and of these 36,652 completed the post-event questionnaire (53.0% response rate).
So, even working with businesses that wanted to play Stepathlon, nearly half the workers didn’t complete the program. Now, think about doing this with businesses and workers who don’t want to play Stepathlon. Think the completion rate would even hit 10%?
Now, let’s pivot back to that weight loss count. It’s based, of course, only upon the 53% who completed pre and post surveys. Just for fun, let’s recompute the weight loss analysis and do an intention to treat count. Let’s be polite an assume that everyone who quit Stepathlon neither gained or lost weight. Now add over 30,000 zeros in the weight loss column and recompute the average weight loss. Hey, without doing any math at all you can easily see that if half the sample lost 3.2 pounds and then we add 0 for each person in the other half who dropped out, the average weight loss would go from 3.2 pounds to 1.6 pounds.
This means that after 100 days of the treatment and counting all who started, not just finished, Other Guys lost about 8 ounces of weight per month on Stepatholon. That’s half a pound of weight loss. Now, that’s still probably statistically significant, but can we open a Windowpane on this? The researchers provide that information in a Table, so with a little effort we can do this. Here’s a screen shot.
Note the box highlight on the right which is the average for all 3 years of the program. And remember this can only include Other Guys who completed both pre and post tests.
First, note that the numbers in the table are different than the numbers in the abstract. Recall the abstract claimed a 1.45 kilo loss on average. If you do the math on the numbers in the table you get 1.1 kilos. What gives? They “adjusted” the weight loss with a variety of other variables. The table reports (I assume) the raw weight loss.
Second, you can see the standard deviations with that +/- notation. I’ll do quick and dirty math and average the pre and post values. That gives an average SD of 12.8 kilos. We can now open the Windowpane on the weight loss. Let’s be optimistic and first use the “adjusted” weight loss of 1.45 kilos; the Cohen’s d is .11, less than half a Small effect, about a 47/53 difference. Now, let’s be honest and use the raw weight loss of 1.1 kilos; the d is 0.08, lesser still than half of Small, about a 48/52 difference. Now, let’s be brutal and go with the intention to treat approach which would add 40,000 zeros, cutting the raw weight loss to 0.55 kilos; the d is 0.04, less than one quarter of a Small effect, about a 49/51 difference.
Let’s put all this together. Start with workplaces who are leaning in on Stepathlon. Count only Other Guys who complete pre and post and ignore the dropouts. You get half a Small effect. So, as long as the Other Guys are already motivated for doing more physical activity, the Stepathlon app ‘n iGizmo appears to produce half of a Small change, but globally. Maybe a little means a lot here, right?
But, we know this is the most optimistic way of counting. Include everyone, even the quitters. Use only the raw self reported weight. Then you get a 49/51 difference on an observational design without any control and based entirely upon self reports.
Call me Poopy, but I’m not sure about this.
Conclusion: Distributed mHealth implementation of a low-cost lifestyle intervention is associated with short-term reproducible large-scale improvements in physical activity, sitting and weight.
I think a hard hearted scientific perspective strongly suggests that Stepathlon doesn’t make any difference. Except for this.
Costs for employee participation were borne by the employers, with the cost of participation modest (approx. $US50 per participant in India, and $US 60 outside India).
So, Stepathlon collected around 50 bucks for each employee and I’m assuming that cost includes not just the completers, but the quitters, too. Do the math on that and Stepathlon collected over $3.5 million dollars over 3 years to operate an app ‘n iGizmo play. There’s no way on God’s good green Earth that it cost Stepathlon $3.5 million bucks to run that website. No. Way.
I think from a cold hearted persuasion panther perspective that Stepathlon does make a difference. On Stepathlon’s bottom line.
Ganesan AN, Louise J, Horsfall M, Bilsborough SA, Hendriks J, McGavigan AD, Selvanayagam JB, Chew DP. (2016). International Mobile-Health Intervention on Physical Activity, Sitting, and Weight: The Stepathlon Cardiovascular Health Study, Journal of the American College of Cardiology.