The Butterfly Ballot Effect
- For the want of a nail, the shoe was lost
- For the want of a shoe, the horse was lost
- For the want of a horse, the rider was lost
- For the want of a rider, the battle was lost
- For the want of a battle, the kingdom was lost.
“And all for the want of a nail.” From tiny inputs, major changes can come. This is often referred to as the butterfly effect, i.e., the flapping of the wings of a butterfly in Beijing can cause a hurricane in Florida six months later. This simple little truth lies at the heart of Chaos theory.
In the 2000 election we saw a demonstration of how this theory works out in the real world. A seemingly innocuous decision to enlarge the type on the ballot in one county in the state of Florida resulted changed the outcome of an election. Initially designed to make the type easier to read for the county's large elderly population, the ordering on the ballots ended up causing confusion among many elderly voters, as well as for those whom English is a second language. As a result, there was a statistical anomaly in that county’s voting that seems to indicate that Jewish and minority voters ended up voting for an avowed white supremacist and anti-Semite. “And all for the want of a nail.”
A member of one party designed the ballot and members of both parties saw the ballot and approved it. The original designer still stands by her design, which is sad. She made a mistake; it cost her former party the election, and helped hand the presidency over to the other party. I can understand why she will not admit she had made a mistake. It is a mistake that should not have happened, however, and wouldn't have happened had the officials in that county tested the ballot before they used it.
User testing is a simple fail safe against the Butterfly effect. It is simply impossible for anyone to foresee how his or her design is going to be used. Users will do all sorts of things that designers never intended, exploiting loopholes and failing to see what the designer thinks is so very obvious. The only way to compensate for this is to test the design with actual users. In the Florida county case, going out to the various communities with the ballot and a booth to let voters try the ballot design would have shown the flaw right away. The design could be changed, and we might have had a different president.
What does this mean for web design? User testing needs to be part of every design process. If it can happen to the United States presidential election, it can happen to your web site. This means more than pulling together a focus group for a few hours.
You need to have an expectation of why users will want to come to your site. What will they try and do? When that is decided you need to ask ordinary people to come in and use your design. Obviously you want to go after the people in the market you are targeting with your site. Pay the people for their time; provide them with some food. Do not give them any initial information about the site or even any help. Let them skip sections if they cannot figure something out. It is important to know what people cannot accomplish. Just sit back and watch them go. You are trying to understand where people have problems with the site.
Take notes where people fail. Pay attention to what doesn't happen. Ask follow up questions after the testing. Let people know you will be making improvements to the site based on their experience. Remember, they are helping you, not putting your design down. Finally, go back to the drawing board and make changes to your design based upon your user input. Then do it all over again.
Usability testing is important to the design process. You need to plan for it from the beginning, not slap it on as a afterthought. It does not have to be expensive, though you will get better results from hiring a professional firm. Without it, your great design is just another good idea, just like the one that put someone else in the White House.