Mobile Usability

To discover what makes websites easy or difficult to use on mobile devices, we combined three usability methods:

• Diary study. 14 participants from 6 countries (Australia, The Netherlands, Romania, Singapore, UK, and US) logged everything they did with their mobile device — aside from making phone calls — for about a week. For each activity, they sent us a Twitter message on the spot; at the end of each day, we sent them a questionnaire to collect more in-depth information.
• User testing. 48 people participated in usability studies using their own phones, and we recorded sessions with a document camera. Half of the participants were men and half were women. The age distribution was fairly even across the 20–49 year range; a smaller number of users were age 50 and above. Of the 48 participants, 33 were in the US (two locations) and 15 were in London.
• Cross-platform review. We conducted a design review of 20 sites, using 6 phones: one feature phone, three different smartphones, and two different touch phones.

In the usability tests, we asked participants to perform typical tasks with their phones. In testing mobile.winespectator.com, for example, we showed people a bottle of wine and asked them to find information about it on the site. For m.lufthansa.com, one of the tasks was: "Your friend is scheduled to arrive in London today around 12pm from Munich, Germany. Find out if her flight is on time."

In all, we tested 36 websites and asked users to attempt particular tasks on each. These site-specific tasks let us systematically observe several users with different phones trying to do the same thing. We also tested 34 Web-wide tasks where participants could use any site they wanted. One such task was: "You and your vegetarian friend want to find a good Indian restaurant nearby. Use the Web to locate one that you may want to go to and that serves vegetarian food." These tasks gave us usability insights into hundreds of additional sites as well as an understanding of how people decide which sites to visit on their mobiles.

The test tasks were inspired by the user activities recorded in the diary studies. The diaries also let us follow user behavior over a longer time period in more naturalistic settings than is feasible for lab studies.