IAM’s Misuse of Dodgy Statistics

Yesterday, there was a furore among the cycling community about a press release that had been issued to the press by the Institute of Advanced Motorists as a result of a SurveyMonkey survey that had been carried out. The survey was placed online and then made public so that anyone could complete it, anonymously (as many times as they liked). The version that is linked above is actually a revised version; the original was even worse.

I will link, at the end of this article, to a few other blogs and reports about this item when I have a bit more time, but am writing here just to report on the correspondence I have had with the IAM about their press release. Here is the conversation:

Me to IAM through their contact web site:

Why is the IAM publishing deliberately misleading headlines about cyclists, based on a dodgy “survey”.

Short and sweet, but to the point I think.

IAM to me (via email):

Thanks for your email.

Can I ask, where you saw this coverage on the cycling poll?

We received complaints on the press release from journalists yesterday morning, and did reissue it with some changes following the feedback that the opening paragraph was misleading.

To see the latest release – which all concerned media have also received – please visit:  http://www.iam.org.uk/news/latest-news/1054-more-than-half-of-cyclists-jump-red-lights

I am sorry you were disappointed by the release, but it is not fair to say the release is based on a dodgy survey. The audience is, as a few people suggested, self selecting, but this will often be the nature of cycling polls and the findings were reported accurately.

If you check the press release on the website you will also see that we did include the primary reasons cyclists do jump the lights, and in the reissue we also added in the body of the release (as opposed to just in the notes to editors) the figures for drivers who do the same.

As the responses from cycling bodies show there is a recognition that there is an issue here and we believe we have stimulated a healthy debate.  The rise in cycling in England is a transport policy success but infrastructure and road user awareness are lagging behind. Our role in improving behaviour is a given but, by highlighting the fact that cyclists run red lights for predominantly self preservation reasons we believe we are contributing to the tidal wave of support for more investment in safer roads for riders and driver alike.

I hope that explains our position,

Well, at least she did apologise, but I still think their article is biased. Here is my further response to them:

Thank you for your prompt reply.

I am aware that you modified the content of your article after receiving complaints about the serious misuse of dubious statistics (see later), and I think the revised article is better than the original.

However, the fact remains that your headline says: “More than half of cyclists jump red lights”. I presume that the choice of headline is a conscious decision in your organization. I can only conclude, therefore, that the choice of this over, for example: “86 percent of cyclists rarely or never jump red lights”, when the latter is a perfectly valid interpretation of your data, was a deliberate attempt to place “spin” on the story. If that is the case, then you have been very successful, as evidenced by the way the press have picked it up, including even the BBC. (The Times has been a creditable exception to this.)

Regarding the quality of your “survey”, as an active researcher myself, before I publish the results of my work, I am expected to conduct a thorough literature review, place my results in the context of other serious research on the subject, and make clear the limitations on the validity of my results based on the research methodology I have adopted. Your article includes none of this “due diligence”, despite it’s being published as a serious study by what I had previously thought was a reputable organization.

Given the nature of your organization, you ought to be aware of the study carried out in 2007 by Transport for London, in which they actually went out and counted the number of cyclists jumping red lights at several junctions in London. You may find the report at:

http://www.tfl.gov.uk/assets/downloads/businessandpartners/traffic-note-8-cycling-red-lights.pdf

Given the serious, scientific nature of this study, the fact that it is a very well-known one, and the fact that it agrees so well with the assertion that “86 percent of cyclists rarely or never jump red lights”, it is surprising that you chose not to refer to its results when making your statements on red light jumping.

Despite your protestations, I still cannot avoid coming to the conclusion that your organization has put a particular “spin” on this story in order to put cyclists in a bad light. If that is not the case, then it shows your organization to be extremely naive in relation to its interaction with the press.

Best wishes,

I did put it a little strongly, but aggressive behaviour from motorists is one of the big problems that I face when I’m just trying to cycle properly on my journey between home and work, and this kind of disgraceful article, along with the press coverage that follows, simply reinforces that problem.

I haven’t yet received a reply from the correspondent at IAM, but she is apparently “Communicator of the Year”, so I am hopeful that she will communicate.

Links to other related articles:

This entry was posted in Article, Road Justice. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *