BAD SCIENCE & UX
And another thing….Books, that may have nothing to do with UX and are still worth a read for UX professionals.
Ever sat in a meeting where two opposing parties – or stakeholders, as you might call them – lambasted each other with what their respective user research had found out? Sure you have, and you may even have witnessed one where they held opposing views on the same usability subject concerning their site: each side using a study they had commissioned to prove their point.
Now that usually is the time when you start thinking about what you can do to help a situation that may not have much to do with users and their experience. But the last time I had that kind of meeting a different thought struck me: how can it be that two studies that are supposed to measure the same thing come to different conclusions? Especially given that it is probably not exactly cutting edge science that we are dealing with here.
I admit: that thought didn’t come from nowhere – at that time I had been reading a book by British writer, broadcaster and doctor Ben Goldacre. The book is called Bad Science and its cover promises fast and powerful relief from scaremongering journalists, pill-pushing nutritionists, flaky statistics and evil pharmaceutical corporations.
The book – in a way based on Goldacres “Bad Science” columns here and in The Guardian – debunks various cases of pseudoscience of the aforementioned varieties of scaremongering and evil. In doing so, it also gives a short and very enjoyable tour of the scientific method: how a proper study should be constructed, how it should be documented, how statistics work, and how peer review is like democracy: it’s not perfect, but the best we have.
Ring a bell? It did for me. It got me thinking about what exactly they had been measuring in their usability studies, how valid the design had been, how transparent the statistics. More often than not it is down to the question that had initially been asked (which gives rise to another interesting bias often found in this context: that of looking for confirmation of a thesis instead of actively trying to falsify it in order to test its validity).
And that’s why I think Ben Goldacre’s “Bad Science” comes recommended for our profession. It’s not only a good read (for most anybody, I guess, but of course even more so if you are interested in science) but it gives some insight into how to check for the validity of the research you are presented.
If you want to go buy it now, here are some links that get you there and help me with this blog:
The English version
Bad Science
Und das Buch auf Deutsch
Die Wissenschaftslüge: Wie uns Pseudo-Wissenschaftler das Leben schwer machen
And here are the links to Ben Goldacre’s blog and column – and to an interesting post on a similar subject I came across just recently.
Bad Science – Ben Goldacre’s blog
Bad Science @ The Guardian
Don Norman on Research and Innovation: Technology First, Needs Last