Month: May 2004

I was listening to the news this morning and they had a piece about how many medical researchers are beholden to the makers of the products they’re testing. Furthermore, the major medical journals who publish this research are sometimes unable to deal with this potential bias. If the makers of Lipitor, for example, are paying researchers to study its effectiveness at reducing cholesterol relative to a competing drug, then that raises all kinds of questions about objectivity. Those questions can be dealt with, of course, and they need not mean that the research is worthless. You’ve just got to have safeguards and full disclosure to everyone, including the readers.

Interesting as all that was, I was more interested in the question of why we don’t do tests of specific products in I/O Psychology. If the medical field can conduct scientific research on name brand drugs and get them published in top-tier journals, why don’t we study off the shelf products used in the area of executive development, selection, and training?

I’m not talking about measuring So-And-So’s Five Factor Model of Implicit Leadership or a meta analysis of studies looking at conscientiousness. I want a team of crack psychologists to study Stephen Covey’s 7 Habits of Highly Effective People training and tell the world if it really does do what it says it does. Let them use lab rats if they need to. I want those same, objective scientists to study the jaunty Impact Hiring system or the use of the dreaded Meyers-Briggs.

These kinds of studies are being done (well, some of them; I’m pretty sure nothing scientific has come within a hundred yards of a Covey seminar), but they’re being done by the test vendors and the consulting firms that sell them. Let me ask you: would you sooner trust a study on the effectiveness of St. John’s Wort put out by Walgreens or one put out by the Journal of the American Medical Association?

I/O psychologists really need to step up and wade in the mainstream more, even if it is polluted.

Ever come up with a great idea for a study and then have someone else beat you to it? I’ve mentioned before how some people morph into complete half-wits when they go online. They just do things that they would never do on the phone or much less face-to-face. Furthermore, I’ve always wanted to study this phenomenon scientifically. Why do they do it? How do we mediate it? People’s stupidity fascinates me when it’s that spectacular.

Well, someone named Michael Tresca beat me to it in a study entitled “The Impact of Anonymity on Disinhibitive Behavior Through Computer-Mediated Communication“. It’s really fascinating reading. He looked at 484 USENET posts (it’s like an Internet message board) and coded them for all kinds of antisocial, nitwit behavior. To quote:

The purpose of this study is to determine if experience with computer-mediated communication will alter a computer user’s behavior and perceptions. Specifically, this study will test the effect of objective anonymity and experience upon disinhibitive behavior in computer-mediated communication.

In other words, they wanted to know why people turn into smacktards once they get online.

Tresca goes on to define the nature of online anonymity and the necessary conditions (e.g., “lack of visual appearance, the flexibility of a label that is different from the user’s normal persona, and relative protection from physical and social repercussions”) for it to exist and impede normal inhibitions. He predicts that what I call the Smacktard Quotient (SQ) should decrease as either perceived anonymity decreases or experience with the ‘net increases.

(Interestingly, he notes that “good writers and more literate people have the same social advantage that physically attractive people have in face-to-face over computer-mediated communications,” which is a point I hadn’t considered that way before. Too bad he doesn’t directly test it.)

Unfortunately, the study’s results don’t pan out. They don’t find strong differences between high, medium and low anonymity and the Smacktard Quotient. Neither do they find differences for experience, though it’s probably because they just measured it with number of posts (thus an Internet veteran who recently joined this particular newsgroup and has only made 5 posts would be counted as inexperienced). The problem I see is that the study uses only one newsgroup as its source of information and measures anonymity from things the users voluntarily adjust (e.g., including a real e-mail address, phone number, etc. in their post).

A better study design would have been to look across multiple groups/boards/whatever, each with higher or lower anonymity requirements. Does a board that requires registration have a lower average SQ than one that allows users to post anonymously?

From a practical standpoint, the research (particularly its underlying theory, which I don’t think was tested well and still makes sense) still suggests a few things that we already know cut down on the level of “inflammatory and informational disinhibition.” Things like:

• Requiring a valid e-mail address
• Account registrations
• Posting I.P. addresses
• Putting personal information on file
• Moderating posters’ contributions until they reach a certain level of participation
• Karma or other rating systems from fellow board members

So while this stuff may get a kind of “well, duh” reaction from most, it IS nice to see it being studied scientifically.

And finally, if you’ll pardon the vulgarity, Penny Arcade summed it up quite succinctly with the following equation: Normal Person + Anonymity + Audience = Total Fuckwad