Eye-Tracking studies- Usability holy grail?

Eye-Tracking- following user eye patterns

Eye-tracking studies are a type of usability test where user gaze concentrations are recorded in thermal-like "heat zone maps". The heat zone maps track user eye movements. Eye tracking tests make usability testing look really interesting, sophisticated, high-tech and scientific. Eye tracking usability data appears to be more valuable or empirical since it is recorded using technology and gaze capture instruments.

The reality is that eye-tracking, while valuable, doesn't make usability testing any more powerful. It's what you do with the observations and the usability test data that counts.

Bottom line: If you are using eye-tracking, to make it meaningful, you must:

1. Have a trained observer or usability professional observing. Eye tracking vendors are not necessarily experts in interpreting usability research. So users looked over there, who cares? What is motivating their gazing activity?

2. Focus on what it is you are trying to learn. What aspect of user behavior are you trying to understand? What will eye-tracking offer that other methods won't?

3. Match what users are actually doing and feeling with the eye-tracking data reports. Data is just data unless it is meaningful and informative.

4. Be aware what eye-tracking is, what types of technologies exist and how your tests should be set up for maximum effectiveness. See the Problems Reported... section of this article below for discussion of this issue.

What Eye Tracking tells us about website usability

One of the recent and big studies to come out this year was the Poynter Institute's "EyeTrack III" 2004 Eye Tracking Study. This is the third eye track study conducted by Poynter since 1991.

Here's what Poynter has found from their eyetracking studies relating to website content usability, page layout, navigation and design: (my comment below finding)

1. Users spend a good deal of time initially looking at the top left of the page and upper portion of the page before moving down and right-ward.
Comment: Another thing to think about is how this user behavior mirrors search engine traffic (i.e. Google Bot visiting your site). Search engines read starting at the top left, and then downward in a left to right column fashion.

2. Normal initial eye movement around the page focuses on the upper left portion of the screen.
Comment: Not surprising when you consider that users are patterned by all the other software and websites that they use which have a standard menu start point (e.g. File, Edit, View...). Note: For Japanese or Arabic it would be the mirror reverse.

3. Ads perform better in the left hand column over the right column of a page.
Comment: The right column is treated by users as an "after-thought" area and should be designed with that in mind.

4. Smaller type encourages focused viewing behavior.
Comment: This is especially true in older or elderly users. For the rest of your users, stick with 9-12 point Sans Serif (Arial, Helvetica, Verdana) with an average of 10-11. FYI: Only developers appreciate miniature fonts!

5. Larger type promotes lighter scanning.
Comment: Most reading behavior consists of skimming and scanning. If you want to slow your users down- use smaller fonts in the body of your content. Use larger fonts to help them cover more territory.

6. Dominant headlines most often draw the eye first upon entering the page- especially upper left of the page.
Comment: Remember, Poynter's focus was a newspaper website. However, bear this in mind for portal type design and intranet design.

7. Users only look at a sub headline if it engages them.
Comment: So make sub-headlines relevant and of course make them the keyword phrases users and search engines use.

8. Navigation placed at the top of a homepage performed best.
Comment: Again, if you understand how users are patterned by other tools they use (Word, IE, Outlook Express)- the goodies are at the top of the page.

9. People's eyes typically scan lower portions of a page seeking something to grab their attention.
Comment: This seems consistent with "Information Foraging Theory" where users are said to hunt for information by "scent" or navigation and content of the highest value to them.

10. Shorter paragraphs performed better than longer ones.
Comment: Attention is clipped on the Internet. Short bursts of attention are the environment you are designing for at all times. Note: Longer product descriptions do better than shorter ones in ecommerce situations. As with all usability findings, context is key.

11. The standard one-column format performed better in terms of number of eye fixations.
Comment: Most users are overwhelmed by the average web page that they try to deflect information as a coping strategy. It is the same phenomenon that occurs at a party when you focus on one conversation and ignore the other conversations around you by categorizing them as "noise".

12. Ads in the top and left portions of a homepage received the most eye fixations.
Comment: Interesting, but I wouldn't recommend putting ads there. *Just because they receive eye fixations doesn't mean they put a smile on the user's face*. This is one of the main points of this article!

13. Close proximity to popular editorial content really helped ads get seen.
Comment: One of the golden "rules" of usability is that anytime you satisfy the user's task (interest, goal, objective), you increase the likelihood or create the conditions that they will be open to other stimuli (advertising, cross-selling etc.)

14. Text ads were viewed mostly intently of all types tested.
Comment: Text ads are popular because they are less distracting, camouflage well with the page and are often not known to be ads and therefore annoyances to the user. Oh, and since Google "pioneered" them- they are the de facto standard in effective web advertising.

15. Bigger ads had a better chance of being seen.
Comment: Also repeat advertising on a page by the same company is being used on many sites to reinforce exposure.

16. The bigger the image, the more time people took to look at it.
Comment: Using larger images (file sizes) is easier these days since 20% or more (USA) are on high speed connections, but using thumbnails with large images is always a safer bet.

17. Clean, clear faces in images attract more eye fixations on homepages.
Comment: Humans are wired to recognize patterns and hard wired to other human faces.

18. Higher recall of facts, names, and places occurred when people were presented with that information in text format.
Comment: Good recall depends on the level of relevancy, good copy-writing and content usability (structure and display).

19. New, unfamiliar, conceptual information was more accurately recalled when participants received it in a multimedia graphic format.
Comment: It is known in the field of cognitive science that the more emotion involved in a learning transaction, the higher the retention and recall.

20. Story information about processes or procedures seemed to be comprehended well when presented using animation and text.
Comment: And the animation or text must be clear, easy to understand and in the language or conceptual world of the audience.

Types of Eye Tracking Technologies

1. Head-mounted tracker: Head mounted tracking devices as pictured at the start of this article (image from Poynter's earlier study) consist of a wire frame helmet that is mounted on the user's head in order to stabilize head and eye-movement.

2. Gaze-detection: This technique in addition to head mounted tracking has been around since the mid-1990's and featured as an interface device in virtual reality research. What's new with gaze detection are technological improvements.

In the Poynter III eye-track study, Stanford University derived  www.EyeTools.com used the new eye-gaze technology developed by Sweden's www.Tobii.se. In this system, the computer screen itself detects, captures and tracks the user's eye gaze patterns. Other vendors like Australia's www.SeeingMachines.com, Germany's www.Eye-Square.com or American www.EyeTracking.com offer headset and headset-free kits. 

Measuring more sophisticated variables...

I expect the next 15 years will see an increase in physiological measurements being used in consumer and usability research. Not because so called "traditional usability" techniques are inadequate, but rather because the field itself will help prove itself with the "hard proof" offered by the new technology. Already Eye-Square offers an additional skin-conductivity sensor to help detect such factors as shifts in sweat, temperature and heart-rate. Eye Tracking Inc. offers pupal diameter measurement as another way to gauge/track emotional response.

Further movements can been seen in  physiological research in Harvard University's lab run by Gerald Zaltman (author of the amazing book How Customers Think), where fMRI (functional magnetic resonance imaging) is being used to determine where data is being processed in specific regions of the brain. Now we're talking "hard proof". It's not what they see but what they think about what they see!

Problems reported with using eye-tracking for Usability Testing

Many eye-tracking firms (and the original companies that were founded) emerged from academic settings and moved toward selling commercial research services. The use of eye-tracking in usability research is fairly new and is recognized to lack empirical evidence regarding its effectiveness.

Eye-tracking technology changes every few years. However, most of the vendor websites do not offer detailed information about their specific technologies or approach (scientific basis, trials or double blind studies with their technology). The attitude seems to be, "eye-tracking is cool, so just do it"!

Take a quick look at the research literature on eye-tracking and a different story emerges.

Schnipke and Todd (2000) at George Mason University reported extensive problems in properly collecting eye tracking data, despite vendor training and one year's experience. They identified a host of obstacles such as ease of use of the system, calibration stability, pupil fluctuations and pupil condition quality as well as the issue of omitting users who wear glasses. The authors used a remote eye-gaze system.

Goldberg et.al (2002) at Stanford University and Oracle Corporation identified two styles of eye tracking studies: top-down (task oriented) and bottom-up (behavioral inferences). The researchers found that both styles of eye tracking studies must be adopted if eye-tracking is to become a routine usability methodology.

Pan et. al (2004) at Columbia University confirmed previous work by Rayner (1998), finding that individual characteristics of the viewer as well as the stimuli both contribute to viewer's eye movement behavior.

Eye-tracking seems to have a promising future. As the technology improves, so too will the research application, methods and action-ability of eye-tracking data. However, eye-tracking does not seem to be the holy grail of usability testing. The two biggest practical problems are calibration and complex reporting and analysis. In corporate usability settings, easy test set-up and quick design insight, guidance and recommendation are the most valuable elements of the usability research activity. If eye-tracking jeopardizes those elements then it looses some appeal.

Scrutinizing the quality and end results of new eye-tracking technology developments and methods will become a bigger problem as traditional academic spin-offs compete for eye-tracking services in a commercial capacity. As with any new technology it is important to remember why you are using it and what it can do for you. The Poynter research like many eye-tracking studies, provides another data point to validate or challenge your existing assumptions about user behavior. As Poynter's Howard Finberg put it, "Eyetrack III is a tool, not a solution".

The reality is that eye-tracking, while valuable, doesn't make usability testing any more powerful. It's what you do with the observations and the usability test data that counts.

Best Wishes,

Frank Spillers, MS

Comments

[Martin L.]

I agree that Think Aloud has the potential to disrupt natural use of an interface. The interesting thing is that I've seen it work in both a booster and drag on usability in lab settings.

Some subjects get distracted talking (and an even smaller number get a little intoxicated on their own opinion!) and lose track of the task they were asked to perform.

Other subjects actually seem to improve their performance by thinking aloud - i.e. "So, I'm thinking that this button here will toggle the thingy to off, but now I say that, it that doesn't really make sense, because the thingy is already off..."
Sometimes articulating thoughts verbally organises thought patterns that were a little disorganised previously.

Having said that - even in the latter situation where the user (probably) completes the task successfully, an observer has learned that the toggle button in question is a potential usability problem. All goes to show that careful analysis of usability results are crucial

[jnh]

It's not where I look, it's what I want. I can sit on Park Avenue and 'look' at hundreds of women---but this tells you nothing about which woman I will approach to ask out on a date. Knowing what I look at only tells you what I looked at. The past does not equal the future.

[Betsy Caporale]

Do you know of any eye-gaze technology that is compatible with Mac OS X?

[Richard Hearne]

Excellent piece and now bookmarked as a fantastic resource.

Thank you :)

[Pozycjonowanie stron]

Great information!

[Dennis van der Heijden]

The Tobii P10 is great for Usability studies... and way more confortable... since there is no need to wear anything!