Your Turn: This Is Not Science

Tack that promises to improve your horse’s movement, comfort and overall ride? Sounds great. Blinding equestrians with “scientific technology” with no actual science to back it up? Not so great. Nicole Sharpe has a bone to pick.

All photos by Nicole Sharpe.

From Nicole:

I saw a post that piqued my interest on Facebook this morning. It was an article about how breastplates can negatively affect the way horses jump. Clearly it did more than just pique — it’s ruffled my feathers enough to break me out of my blogging ennui. Thank you, bad science!! You’re just what I needed.

So let’s take a look at this article. I’m not going to blame the reporters too much for this write-up, since they aren’t science reporters and are really only able to work with the information they are given. You may notice that the first suggested link after the article is about a girth “scientifically proven” (every scientist’s favorite two words) to improve the way horses go.

The TL;DR of this situation: this is not science.

There are so many glaring red flags in this “study” that I can hardly list them.

There is no link to a peer reviewed journal article, or any data, figures, or any other “scientific” measures of difference. Not even a mention of intention to publish (though, to be fair, the manufacturer has several peer-reviewed journal articles regarding other equipment studies they have done). The article does point out that the horses took off “closer” (no measurement) and landed “closer” (no measurement) to the fences, thus increasing the flexion of and strain on their hocks and other joints (no measurement). But there are literally no measurements in the article – are we using meters, feet, centimeters, atoms… ?

Obviously the breastplate made him do it.

The “researchers” (the tack manufacturer) even provided a handy-dandy little image that demonstrates how different the arc of the horse is. No matter that in the “better” image the horse has already started to take the forward part of the landing stride with his front feet and that is from where they measured his “landing” point, and in the “bad” image the horse is pictured at a different part of the landing phase, and his landing point is measured from the foot that is further back. Plus we all know that every horse jumps every fence the exact same every time, and nothing but equipment ever influences this — not rider balance, approach, speed, or general attitude on the day!!

All’s fair in marketing and “research,” right?

The breastplate made him do this too.

And what about those oh-so-critical study numbers that people are always reporting? Things like sample size, p-value, effect size, or even the dastardly value of measurement? “Significance” (their scare quotes, not mine) is all well and good, but if the effect size is less than 1%, you’re going to have to work pretty hard to convince me that your product method has any real relevance. Wow, excellent, I can improve my horse’s bascule by less than 1% by spending $350 on your special piece of equipment. Talk about promoting a quick fix.

There is that upper-level rider’s testimony. He says his horses jump so much free-er in front, and he can feel it. But really, humans are biased and fickle things, and just because we think something is true doesn’t mean it is true. Especially not subjective, un-measurable things like the feeling of a free-er jump that might be influenced by free product or small piles of gold coins.

It’s probably just me, but I am sick and tired of the lack of science that goes on in the equine industry! We all want to do the best for our horses, and I get that. But a little bit of testing, common sense, and critical thinking goes a LONG WAY with this stuff. Fortunately, many readers of this article already figured that out.

And finally, one additional pet peeve: If it’s real science, nobody is ever going to use the phrase “scientifically proven” to talk about it. Scientists don’t use those words because science is always changing and adapting.