Demystifying Usability

Web 2.0: A glimpse into the future?

What is Web 2.0? (as of Dec. 2005)

A new approach to web and web application development that is characterized by:

• Web sites offering free (cool) web services and tools (examples below);
• Open source code (open API's)- software code becomes a viral marketing tool;
• An enhanced user experience afforded by new approaches to information sharing and use of existing/new web scripting technologies;
• Giving users the ability to share their opinions, media or to offer back to the community aka "The Architecture of Participation" (see Wikipedia's Web 2.0 definition) or peer-to-peer goes mainstream.

Web 2.0 seems to have taken flight in the last 10 months related to:

• The rising popularity of social networking websites (like MySpace); online dating sites (like Match.com) and social networking sites (like LinkedIn) that create more emphasis on the "Social Web".
• Google's rising influence and use of new ways to present a map interface (Google Maps; Gmail; Google Suggest). And Amazon, Microsoft and other big players are working on what will be central to the "Web 2.0" user experience (i.e. Tagging in Amazon and RSS web feeds in IE 7).
• Yahoo's positioning as a stakeholder in Web 2.0 with it's acquisition of Flickr photo sharing and del.icio.us social bookmarking services and hiring of key "Social Media" executives to spearhead the new effort. "Sharing will be everywhere," said Jeff Weiner, a Yahoo senior vice president in charge of the company's search services. "It's the next chapter of the World Wide Web". FYI: Fortune magazine ranks Yahoo! as #1 in 100 Fastest Growing Companies with a 78% revenue growth rate.
• The shift in the one-way broadcast or publish model to a two-way flow of information where consumers are active creators in content aka User-Contributed Content (Experience Dynamics web seminar)....or what former US Vice President Al Gore calls "We Media" (see We Media 2005 Conference website).
• A new momentum largely coming out of San Francisco; largely made up of former dot-com enthusiasts (sober but dreamy)- and a significant amount of Venture Capital dollars financing  Web 2.0 infrastructure.
• An annual conference called "Web 2.0"(in it's second year) sponsored by O'Reilly Media (see Tim O'Reilly's What is Web 2.0?).

A word about hype

Is Web 2.0 just hype? Many large organization including Yahoo!, Google and Microsoft are investing heavily in the technologies and projects that are driving Web 2.0. If you loathe hype (as I do), perhaps you should see it for what it is: Hype seems to be an expression of the excitement generated by something new; it is often careless, clumsy and poorly thought out- but it's real purpose is to excite and ignite the feeling of change, progress and newness.

Having said that, I think this interesting article by Russell Shaw at ZDnet, is refreshing in terms of putting the Web 2.0 "hype" into context:

"Web 2.0 is bunk. Not that the elements of this rebirth aren't there. I write about some of them, and Richard has them nailed. It's just that they cannot be classified under a common umbrella. They are forward lurches of various standards and technologies, some compatible, some not. Some revolutionary, some evolutionary, some impractical. Some are collaborative, others are highly competitive with each other".

What's New with Web 2.0 as a technology shift?

What's different about this technology curve is that this time, user experience is on the agenda, despite the fact that Amazon's Jeff Bezos sees Web 2.0 as "about making the Internet useful for computers". (Amazon's focus is on creating viral API's with Amazon Web Services). To give Bezos credit, he also has said,though not in his Web 2.0 related interviews,...

If there's one reason we have done better than most of our peers in the Internet space over the last six years, it is because we have focused like a laser on customer experience, and that really does matter, I think, in any business.

Some Dominant Web 2.0 Themes

1. "Mash-Ups"- using others code (e.g. many are using Google Maps) and sharing your own code (e.g. Amazon is trying to provide code) to create a new application or web service such as: London Terrorist Attack map; eBay Motors & Google Maps; Zvents events site; Trulia real estate search;

2. "Tagging"- letting users create categorization on the fly with "tags". The goal is to help create more personalized, context rich search and categorization. Encouraging users to tag, like Amazon is doing currently, helps create better search. Some sites that tag or promote tagging: TagWorld; Tag Cloud; Technorati; 43 Things, del.icio.us (Yahoo!).

3. "User-Contributed Content"- letting users comment, rate, add content or media such as photos, videos etc. Becoming a standard in online newspapers and blogs. Some sites that promote user-contributed content: Judy's Book, Insider Pages; PriceGrabber; IRV2; BBC News, TripAdvisor.

Even conversations are occurring with regard to using the social web to make advertising more relevant.

How Users and User Experience Will Benefit from Web 2.0

1. More Competition Will Improve the Overall Web. Free services and an open source model that is going mainstream will mean more improvements aimed at remaining competitive. Google Maps and Google GMail (two "Web 2.0" applications) have pushed mapping and free email services into over-drive, not to mention desktop search.

2. Better Web User Interfaces Will Improve Subjective Satisfaction. Potentially being able to do more natural interaction (such as drag and drop or zoom) will give users a more intuitive way to interact with data. A focus on sharing will make the web seem more close to home or to a social circle (existing or created). Tagging will potentially create a user-centered "information architecture" that relies on context, semantics and personal labeling systems or so-called "folksonomies" rather than hierarchies or taxonomies.

3. More Trust and Credibility from Institutional Sources. Allowing self-publish or two-way publishing will give users more control, power and choice of data and opinions, views and decision-making resources. Blogs and ways to better integrate or share text, video and audio blogging will change what it means to be informed about a purchase decision or a visit to a destination.

Bottom Line: Web 2.0 seems to be reflecting the changes in the world and the way the current generation of web innovators interpret and re-purpose existing and newly used technologies. Web 2.0 or whatever you call it, is already here. For example Public Relations and Marketing savvy people use del.icio.us to monitor buzz; Technorati to track blog feeds and Epinions to monitor customer criticism, wants and needs. Pay attention to what is happening in Web 2.0, because it represents change at a micro and a macro level and whether you notice or not, change will affect your site whether you like it or not.

Best Wishes,
Frank Spillers

Good Follow Up Reads:

Web 2.0 and the Long Tail (Chris Anderson- Editor in Chief WIRED magazine)

Relax, Everything is Deeply Intertwingled (Adam Rifkin)

Are you ready for Web 2.0? (BayCHI panel -Audio program)

Configuration Hell- The Case for the Plug and Play User Experience

You know the drill: You download or install a new piece of software or open a new piece of technology (e.g. PDA, mobile phone, laptop) and you have to "configure it" to get something to work or work the way you want it to...

Summary: Users are not usually successful at configuring software, websites or devices and the configuration experience can be a major source of frustration. Instead we need to move toward a world where everything is auto-configured and user experiences are "plug and play".

Defining Configuration: I refer to configuration in it's broadest sense: A user must perform some advanced action in order to get some desired result from software, a web site or a product experience. Typically the term configuration means to adjust or change settings- hiding, activating or altering useful features and system behaviors.

Basic Examples: You want to change your desktop picture?- You need to find that right click menu and select your new choice. You want to use flash on your digital camera at night?- You have to scroll through the menus and select that option- (don't forget to turn it off!) You want to re-connect to a new wi-fi (wireless) network or a secure network, you need to view available networks, choose one or add an encrypted key. Need to watch a video online?- You need to know your connection speed and choose your player.

Note: If these examples are no-brainer tasks for you, then you are probably deep in the forest of configuration and configuration is second nature to you. This article explains why you might not be able to relate to why configuration is such a major usability issue for average users (Average users = people who don't work with computers for a living).  

Symptoms of "Configuration Hell"

1. On the web. A shopping cart is a type of configuration experience. When you remove an item and forget to update the cart, your cart will be inaccurate. Playing with a poor shopping cart is a symptom of configuration hell. Navigation usability issues can be seen as a type of configuration SNAFU in the sense that users are trying to steer a website in a particular direction to serve up the desired pages or functionality.

Changing an interface from current state to desired state is the basic unit of the task of configuration.

2. Updating Your Account. Amazon's lousy Account Management user experience (which never seems to get any better) is another example. Ever notice how hard it has been over the years to get  information on an order at Amazon? So difficult, they put an "animated demo" of the Account section in there to "help" (see this article about how Help, Never Does).

3. Mobile Devices. Tried setting up email on your mobile phone (smartphone or PDA) lately? It can be a total nightmare depending on your phone brand/model.

4. Operating Systems. No finer example. The entire Operating System user experience (Windows, Mac and Linux) is a big configuration love fest. Installation and in particular driver installation are artifacts of the legacy of software configuration (Yes, configuration for end-users is part of the past, and not part of the future, I believe. The big shift in Windows came with auto-detection of Media devices and Wireless networks in Windows XP in 2001). Ever tried going online with a Windows 2000 laptop and your wireless card? It will boil your blood.

Lindows has tried to address the OS software installation experience (an aspect of configuration) with near instant installation of the entire Operating System  ("seven minutes" in a recent Extreme Tech review). Another innovation is the streamlining of software installation for the OS with the Click-N-Run functionality (same review next page):

"Click-N-Run - slick, slick, slick!
One of the most annoying things for newbies about installing software in Linux is that most apps don't even give you an icon to click to start them after installation - and it's hard to find those applications in the first place. Experienced Linux users know where to go, and can easily create their own icons, or use the command prompt to start new applications. But Linux newbies aren't used to that and sometimes can't figure out how to find or easily start the application they just installed.

[FS: It's not just a Linux phenomenon! If you need evidence of this, visit a relative and ask them if they are using their printers/scanners/digital cameras etc. You're sure to find issues with driver installation or configuration literacy.]

Thus Lindows has a Click-N-Run feature that makes it simple to find a new application, then download, install and run that application. It all happens with a simple click of the mouse. We've got to give the Lindows people credit; Click-N-Run is well designed and easy to use. It's pretty much a no-brainer to navigate the Click-N-Run store to choose software. And after installation Click-N-Run gives you the option of starting the software, adding a link to it on your desktop or adding it to Auto-start. While this might seem like an unimportant detail - it's not. In fact it's very important".

The Mac has a similar function with regard to installation where .exe files (or .dmg files as they are called) are sometimes drag-able to the Applications folder causing them to be instantly "installed".

5. Browsers. Managed your cookies or Java settings lately? Okay, how about 5 years ago? Spyware these days has more people clearing cookies and cache files- but remember how your co-workers didn't even know how to do that a few years back?

Configuration usability issues leave a trail behind: see this user's helpful tips. The Firefox browser made some leaps and bounds with "browser switching", a technique borrowed from a new trend in banking- the "we'll help you defect" switch kit...

When installing Firefox, it pulls all your Internet Explorer settings over, including bookmarks and cookies to the browser. It's fairly slick and worth noting. The Firefox folks at Mozilla also seem to be aware (article: Realities of Users) of the truths inherent in this posting you are reading now ;-)

6. Blogging software. (Typepad, Moveable Type). Need to show your bio? Add an XML feed? Change your design template? All configuration tasks. Typepad has won an award for making it easier, but still managing these important configuration settings can be a little confusing.

7. Search Engines. Google eliminated the search configuration paradigm, removing the need for users to pre-select the Boolean filters AND/OR/NOT (drop-down menus and radio buttons). Simplicity of search interface was a raging battle at search engine companies for years until Google cleared the noise. However, many website search interfaces are obssesed with letting user's fail with Boolean filtering. (See this related post on Site Search usability for more detail).

8. The List is Endless: Home Theater Systems, Camcorders, Cameras, Web Analytics software, Content Management Systems, Email Marketing Software, Document Management Systems, Spyware Remover software, Firewall software, Anti-Virus software, Instant Messaging Clients, Telematics (Automotive) systems...the list goes on...

Towards Universal Configuration in Design

Why do we have configuration in the first place? The configuration mentality comes to us by way of legacy computer systems and legacy engineering-centered designs.

Configuration is the design decision that says:

• "We don't know what the perfect default should really be".
• "We want to let our users be the final judge".
• "Users can go into these advanced settings to change things".
• "To begin with or to add additional power to the functionality- configuration must occur".

I believe that user experiences that force configuration will become unpopular over time. Rather than exposing users to DIY interface engineering, we need to give them transparency, seamlessness, elegance. Your users should walk up to it and synch! Your users should open it up and be greeted warmly before being transported to their destination. Your system should auto-detect, auto-configure, auto-respond, auto-heal.

Don't let your users play with your brilliance, just let them experience the value they seek.

Why is universal configuration imperative? It relates to appropriately using the interaction design technique of progressive disclosure and also the importance of selecting defaults carefully (the topic of a future post).

Also there is the reality of user behavior. Here's how non-technical users relate to configuration:

1. "Stop it, I don't want to Configure Anything!"

2. "What is configuration?"

3. "Why do I have to configure it?"

4. "What is the best way to configure it?"

5. "What is the fastest way to configure it?"

6. "I didn't know I had to do that".

7. "Why isn't it already set up for me?"

 Universal Configuration is Coming!

Rather than giving users the bitter taste of choosing options and making choices about display views, system settings and feature access-- many manufacturers and application developers are giving users a plug and play user experience. I recently purchased a Toshiba notebook that boasted hasstle-Free "Config-Free" Connectivity on the outside box.
                                                                 

I need to tell you, my expectations were racing- was I about to be embraced by a flesh and blood example of what I call Universal Configuration (auto-configuration everywhere on every thing)? No! The Toshiba laptop was not config-free. It had more configuration gymnastics associated with it than any other laptop I have set up in the last five years. It's problems started with the never mentioned and unique hard key wireless lock switch (defaulted to "lock" or wireless "off" mode). To simply go online with Windows XP, I had to involve technical support and it took several hours of trouble-shooting!

Bottom line: We need the mentality of Universal Plug and Play and Universal Configuration in design and development, period. You should open a new laptop in the future and have simultaneous config-free access to wireless, ethernet LAN and dial-up connectivity without any need to manually configure. Windows XP wireless network auto-detection and configuration is a great leap forward. However, have you ever tried configuring a dial-up connection from a hotel lately? If you haven't done it in a while, be prepared for a "configuration hell" experience.

Best Wishes,
Frank Spillers, MS

The Politics of Pop-Ups, Pop-Up Blockers and the Pop-Up Error Message

Pop ups are dead- so what?

First of all, let me just get this out in the open: there is nothing wrong with pop-ups per say.

Next, let me qualify what I just said: Pop-ups are ineffective as an advertising tool and are not recommended on the web since pop-up blockers are ubiquitous. E.g. Every browser and search engine tool these days has a pop-up blocker: AOL pop-up blocker, Google Toolbar pop-up blocker, Yahoo Toolbar pop-up blocker, Firefox built in pop-up blocker, third party free pop-up blocker tools...etc.

Google Killed the Pop-Up

From the late 90's to the early 00's Google quietly used it's simple and easy to use design (read great user experience) to hijack the pop-up as a form of web advertising. If you remember every credible large and small website used pop-up windows for advertising and more. Google's text advertising (right column block short phrase with a link) wiped out the pop-up since the click through rates (CTR) for text based advertising (or contextual advertising) were higher: under .5% CTR for pop-ups to 2-3% CTR for text ads.

Back to "There is Nothing Wrong with Pop-ups"

Pop-ups are not the problem. It is the forcing of them on your users without telling them, which also violates the permission-based marketing model. This issue is rarely understood I find due to the culture of pop-up hating. Yeah, I hate them too.

However, if it is appropriate to provide more detail such as help text, or larger images or diagrams- then by all means use a pop-up dialog! Just be sure that it is self-selected. In other words the user clicks to activate. Remember, pop-ups became hated because they forced themselves on users, got in their face, redirected their attention and cluttered up their train of thought.

The Pop-Up Blocker Error Message

Alas, with so many pop-up blockers (thank you to all the pop-up blocker tools) there is a need for a new kind of error message to help users orient to that which is supposed to be happening which isn't happening.

You've done it right? Clicked on a link, page, image and then...(nothing). What happened? Oh it's the pop-up blocker that ate it. So while you figured that out, the average user who is not savvy to browser features and functionality (such as pop-up blocking) will probably blink and miss the hidden "aha"...and then move on.

(Click image above to see an example of a good pop-up blocker error message)

The American Automobile Association website has a great example of a user-friendly pop-up blocker error message. Bottom line- anyone using pop-ups on their site needs one. Leaving the recovery from a pop-up blocker's work to chance is not enough.

Usability Guidelines for Using Pop-ups?

So if you must use on of those pesky pop-ups (either self-selected or forced) be sure to follow the following usability guidelines for pop up dialog windows:

1. No intensive scrolling. Pop-up information (tables, images, text) should be short and sweet. e.g. 150-200 words max.

2. No left to right (horizontal) scrolling! Pop-ups are a limited attention adjunct to information. They serve the purpose on the web of the ellipse (...). We have seen pop-ups used that have long scrolling tables, buttons down, left, right, outside of easy reach. Our usability testing experience shows that users hate to scroll your pop-ups.

3. Tabs work well with Help Pop-Ups. A few tabs to help organize HELP information seems to work well. Again be careful the length of content. If you have that much to say, say it on a web page.

4. Use the pop-up blocker error message (above). No brain-er, but rarely used. Nice work AAA!

5. Let the user self-select a pop-up always. Unless you are doing some kind of intercept (for a survey or exit email capture offer).

So, next time someone says "I hate pop-ups" and everyone grumbles and says "yeah, me to" you can smile and say to yourself "yeah, but..."

Now, let me go close some of those pop-ups ;-)

Best Wishes,
Frank Spillers, MS

How to avoid being blinded by your own design: Seeing the Forest for the Trees

Whom this applies to: Designers, Marketers, Developers, CEO's

If you design something for your company, organization or department, or help influence the direction of a design, it regularly can become very difficult for you to separate yourself from the design. And chances are, you are not even aware of it most of the time!

This entry looks at why this seems to happen and what you can do about it (if anything at all).

Identifying the problem

One possible answer as to why we loose objectivity when we create or contribute to a design is rooted in the Gestalt Psychology phenomenon of figure and ground:

The phenomenon of figure and ground in perception has been explored extensively by gestalt psychologists. A classic example is that of a picture that either appears to be a light colored chalice on a dark background, or two dark faces against a light background, depending on what aspect of the picture is focused on as ‘figure’ and what is perceived as ‘ground’. (see Figure 1)

                               

Figure 1: The "Vase Faces" illustrating the "Figure-Ground" phenomenon. Is it a face or a vase?

The closer you get to an object (figure) the more blurred it becomes (ground). Figure/ground remind us that perception is relative and not absolute.  Or the more time you spend in internal company meetings discussing a design, the more blurred your objectivity becomes.

It's a symptom that is probably responsible possibly for 95% of poor usability design choices.

Let's call it Heisenberg's Rule of Design: The closer you are to a design the less objective you become.

The more precisely the position is determined, the less precisely the momentum is known in this instant, and vice versa. --Heisenberg, uncertainty paper, 1927

When knowing "too much" can blind you

Every designer goes through this process of becoming consumed by his or her design ideas and assumptions dictated by style, taste or personal preference when creating the look and feel of an application.

Every developer experiences this when he or she tries to "skin the UI", code the GUI or add the User Interface to an application after a long day of coding.

Every marketer experiences this when he or she tries to map new features, new ideas, new ways to engage the customer to the functionality requirements.

Every business analyst experiences this when he or she tries to specify requirements based on business processes, system responses and user/group work flow.

Every VP or CEO experiences this when he or she drops in on the design team and projects the original vision, strategic direction, or business needs onto the design (mixed in with a little personal preference or as Jeroen van Erp put it at last year's Design and Emotion conference, design can be directed by "the CEO's wife").

To figure out how our perception blinds us, let's look at the stages of this "Forest for the Trees Syndrome"...

The Stages of "Forest for the Trees" Syndrome

Translation for International Readers: "Forest for the Trees" means you loose sight of seeing the "big picture" in something, because you are too close to the details.

Stage 1: Attached to the design

During this stage you become attached to your design. This is typically caused by spending too much time with the design and refinements. In a sense the design becomes a part of you and you necessarily feel like defending it because it makes sense to you.

Motto: "I don't see anything wrong with it".
Action: Argue for the design.

Stage 2: Blinded by the design

During this stage you are so exposed to the design (company objectives, brand, issues, constraints, history) that you can't even see that you are biased. Having argued for the design, you are now completely bought into it and are completely blinded from any other information.

Motto: "This is the only way to go".
Action: Fight for the design.

Stage 3: Hypnotized by the design

During this stage, you are so far gone the design has become second nature- like the furniture in your office. You don't question, you don't even think about it or feel that anything is wrong. You can't look at the design with a fresh set of eyes either because you are too patterned from over exposure or by now it seems perfectly fine or justified.

Motto: "This way seems normal".
Action: See any criticism as unfounded and unfair.

Is there a light at the end of this tunnel or are we stuck with tunnel vision?

The field of Human Computer Interaction (HCI) where Usability, Information Architecture and User Centered Design fall out of- represents a way out. User-centered Design (aka UCD), combines a set of methods, techniques and approaches that creates more objectivity in design by leveraging user data, user needs, user issues, user insights and user advocacy. User-centered design is a methodology (popularized by Donald Norman e.g. see his early book User Centered System Design) that triangulates technology (systems) and marketing (features) centered approaches with an outside look at what the user wants and needs, expectations, desires and requirements.

The User Centered Design approach (an industry standard usability methodology) provides several techniques to help "see the forest for the trees". From a usability standpoint, the forest is the  user group. The trees are the features that sit between the application architecture and the user.

What does User Centered Design do that helps bring more objectivity to a design: (or at least ways we have found to help our clients at Experience Dynamics at leading user centered design firm based in Portland, Oregon):

Usability reviews: Analyzing a design from the perspective of users and their tasks with best practices (research based)
Outcome: Advocate for user needs around confusing, annoying, frustrating or difficult to use design elements in order to make better decisions about the direction of the user experience.

Usability testing: Having customers assess a design to detect confusion points and uncover areas of the design that mismatch their expectations.
Oucome: Bring user verbatim feedback from usability testing data direct to the design room.

Field Studies: Going to the user's natural environment and observing their world: seeing, hearing and feeling what they think, want, need...and learning how they construct and prioritize experiences.
Outcome: Incorporate research-based customer personas into the interaction design by seeing how Persona "X" or Persona "Y" will use the design.

Is User Centered Design a sure fired way to prevent seeing the forest for the trees?

No. Especially not with your own design. That is what motivated me to share this with you. Every time I work on a design for my own company (Experience Dynamics) I run into "Heisenberg's Rule of Design" or "Forest for the Trees Syndrome". At this point I know it's:

• Time for a second opinion
• Time to get user feedback
• Time for a break
• Time to advocate for the user
• Time to stop seeing trees and get back to the forest
  

Which reminds me to ask, is the glass half full or half empty?

Best Wishes,
Frank Spillers, MS

How many users should you test with in usability testing? (Latest Research)

Question: How many users do you need to test with for a usability test?

Answer 1: = 5 users (Jakob Nielsen and Thomas Landauer, 1993).

Answer 2: = 15 users (Laurie Faulkner, 2004), PDF file.

So, which is it, 5 or 15? And why are we arguing about an extra 10 users, doesn't one need to test with at least 100 or more users for statistical significance, accuracy and validity?

Statistical Validity in Usability Testing

Usability research is largely qual-itative, or driven by insight (why users don't understand or why they are confused). Qual-itative research follows different research rules to quant-itative research and it is typical that sample size is low (i.e. 15 or 20 participants).

The end result of usability testing is not statistical validity per say (the outcome of quant-itative research) but verification of insights and assumptions based on behavioral observation (the outcome of qual-itative research).

Why don't we do large numbers in usability testing?

1. We are looking for behavioral based insight (what they do).
2. Statistics tell half the story and often are devoid of context (e.g. Why did they fail?)- Also one of the major problems with gaining insight from web analytics (website traffic statistics).
3. Our objective is to apply findings to fix design problems in a corporate setting (not academic analysis).
4. Research shows that even with low numbers, you can gain valid data.
5. Usability testing is being used industry-wide and has been for past 25 years. Experts, authors and academics put their reputations and credentials behind the methodology.

Behavior vs. Opinion

Usability research is behavior-driven: You observe what people do, not what they say.

In contrast, market research is largely opinion-driven: You ask people what they think and what they think they think. You need big samples for market research because of this (though focus groups bend this because they are somewhat qualitative). This is why phone or web surveys require hundreds or thousands of responses. Behavior-driven research is more predictable. Basically, if 10/15 users are confused you can assume that many more will also be confused as well.

Example: If you ask someone "what do you think of this homepage?", you will need several hundred responses to gain statistical validity in order to validate what will be opinion-driven data. Asking someone their opinion does not constitute usability requirements, since usability testing is about isolating "how they will actually use" the design not just "what they think" of the design.

If you give a small set of users a scenario that forces them to interact with home page elements and observe their behavior, and listen to their unsolicited reactions, you will get a better idea of what they think and need. The driver here is expectation (governed by cognitive factors) vs. opinion which can be driven solely by emotional, social or personal factors.

Suggested Sample Sizes for Research

Corporate Usability Research:

• Surveys (phone and web) = ~240-~1,000 +
• Focus Groups = 15-20 (depends on audience segments involved and goals of study)
• Usability Testing = 10-15 participants
  
• Field Studies = 15-40 participants
  
• Card Sorting = 15-30 (higher is better since card sorting uses the statistical method of cluster analysis)

Academic Usability Research:
Samples are usually larger depending on size and scope and research objectives (e.g. 15 users per segment or 40-100 users in a usability test).

Jakob Nielsen's "test with 5 users" assumption

I think it is important to understand that Jakob Nielsen was trying to promote usability testing as a regular usability research activity in corporate environments. I believe he conducted this research (using a call center software application in the early 90's, rumor has it) in order to demystify the perceived complexity of setting up and running a usability test.

Remember in the early 1990's, only the hard core research and development labs at Apple, Bell Labs, Microsoft, IBM and Sun were doing usability testing. In Nielsen's much respected and equally criticized article "Why You Only Need to Test With 5 Users" (written in 2000) he recommends (based on the early 1990's analysis) that instead of opting for higher accuracy, you go for the "fast and dirty" approach of conducing three tests instead of one "elaborate" study.

Later on in the article Nielsen says that the rule only applies if your users are comparable. If you have other segments or user types, you will need to test more users.

Translation: 5 users per audience segment or target user group, or for a website with 3 diverse segments you will need 15 users for the one test.

Magic Number 15 for Usability Testing...or Why 5 Users is Not Enough

Laurie Faulkner ( PDF: 2004) has conducted new empirical research showing benefits from increased sample size.  In her study, "Beyond the five-user assumption: Benefits of increased sample sizes in usability testing", she wrote:

It is widely assumed that 5 participants suffice for usability testing. In this study, 60 users were tested and random sets of 5 or more were sampled from the whole, to demonstrate the risks of using only 5 participants and the benefits of using more. Some of the randomly selected sets of 5 participants found 99% of the problems; other sets found only 55%. With 10 users, the lowest percentage of problems revealed by any one set was increased to 80%, and with 20 users, to 95%.

At Experience Dynamics, (usability consultancy) we have found that the cost savings of using fewer users is negligible. In other words, after you spend the time and money to set up, facilitate and report on the test, adding a few more users does not add "that much" time and money to the overall project.

The benefit you get from adding a few more users to the total (or in the case of 5 users, doubling the amount) is far greater than the small test that gives you "quick and dirty" results. In the case of running a series of usability tests or iterating your testing process (recommended for refinements based on evolving design decisions), you may want to choose a smaller number of users: I recommend no less than 8 users.

Best Wishes,
Frank Spillers, MS

Eye-Tracking studies- Usability holy grail?

Eye-Tracking- following user eye patterns

Eye-tracking studies are a type of usability test where user gaze concentrations are recorded in thermal-like "heat zone maps". The heat zone maps track user eye movements. Eye tracking tests make usability testing look really interesting, sophisticated, high-tech and scientific. Eye tracking usability data appears to be more valuable or empirical since it is recorded using technology and gaze capture instruments.

The reality is that eye-tracking, while valuable, doesn't make usability testing any more powerful. It's what you do with the observations and the usability test data that counts.

Bottom line: If you are using eye-tracking, to make it meaningful, you must:

1. Have a trained observer or usability professional observing. Eye tracking vendors are not necessarily experts in interpreting usability research. So users looked over there, who cares? What is motivating their gazing activity?

2. Focus on what it is you are trying to learn. What aspect of user behavior are you trying to understand? What will eye-tracking offer that other methods won't?

3. Match what users are actually doing and feeling with the eye-tracking data reports. Data is just data unless it is meaningful and informative.

4. Be aware what eye-tracking is, what types of technologies exist and how your tests should be set up for maximum effectiveness. See the Problems Reported... section of this article below for discussion of this issue.

What Eye Tracking tells us about website usability

One of the recent and big studies to come out this year was the Poynter Institute's "EyeTrack III" 2004 Eye Tracking Study. This is the third eye track study conducted by Poynter since 1991.

Here's what Poynter has found from their eyetracking studies relating to website content usability, page layout, navigation and design: (my comment below finding)

1. Users spend a good deal of time initially looking at the top left of the page and upper portion of the page before moving down and right-ward.
Comment: Another thing to think about is how this user behavior mirrors search engine traffic (i.e. Google Bot visiting your site). Search engines read starting at the top left, and then downward in a left to right column fashion.

2. Normal initial eye movement around the page focuses on the upper left portion of the screen.
Comment: Not surprising when you consider that users are patterned by all the other software and websites that they use which have a standard menu start point (e.g. File, Edit, View...). Note: For Japanese or Arabic it would be the mirror reverse.

3. Ads perform better in the left hand column over the right column of a page.
Comment: The right column is treated by users as an "after-thought" area and should be designed with that in mind.

4. Smaller type encourages focused viewing behavior.
Comment: This is especially true in older or elderly users. For the rest of your users, stick with 9-12 point Sans Serif (Arial, Helvetica, Verdana) with an average of 10-11. FYI: Only developers appreciate miniature fonts!

5. Larger type promotes lighter scanning.
Comment: Most reading behavior consists of skimming and scanning. If you want to slow your users down- use smaller fonts in the body of your content. Use larger fonts to help them cover more territory.

6. Dominant headlines most often draw the eye first upon entering the page- especially upper left of the page.
Comment: Remember, Poynter's focus was a newspaper website. However, bear this in mind for portal type design and intranet design.

7. Users only look at a sub headline if it engages them.
Comment: So make sub-headlines relevant and of course make them the keyword phrases users and search engines use.

8. Navigation placed at the top of a homepage performed best.
Comment: Again, if you understand how users are patterned by other tools they use (Word, IE, Outlook Express)- the goodies are at the top of the page.

9. People's eyes typically scan lower portions of a page seeking something to grab their attention.
Comment: This seems consistent with "Information Foraging Theory" where users are said to hunt for information by "scent" or navigation and content of the highest value to them.

10. Shorter paragraphs performed better than longer ones.
Comment: Attention is clipped on the Internet. Short bursts of attention are the environment you are designing for at all times. Note: Longer product descriptions do better than shorter ones in ecommerce situations. As with all usability findings, context is key.

11. The standard one-column format performed better in terms of number of eye fixations.
Comment: Most users are overwhelmed by the average web page that they try to deflect information as a coping strategy. It is the same phenomenon that occurs at a party when you focus on one conversation and ignore the other conversations around you by categorizing them as "noise".

12. Ads in the top and left portions of a homepage received the most eye fixations.
Comment: Interesting, but I wouldn't recommend putting ads there. *Just because they receive eye fixations doesn't mean they put a smile on the user's face*. This is one of the main points of this article!

13. Close proximity to popular editorial content really helped ads get seen.
Comment: One of the golden "rules" of usability is that anytime you satisfy the user's task (interest, goal, objective), you increase the likelihood or create the conditions that they will be open to other stimuli (advertising, cross-selling etc.)

14. Text ads were viewed mostly intently of all types tested.
Comment: Text ads are popular because they are less distracting, camouflage well with the page and are often not known to be ads and therefore annoyances to the user. Oh, and since Google "pioneered" them- they are the de facto standard in effective web advertising.

15. Bigger ads had a better chance of being seen.
Comment: Also repeat advertising on a page by the same company is being used on many sites to reinforce exposure.

16. The bigger the image, the more time people took to look at it.
Comment: Using larger images (file sizes) is easier these days since 20% or more (USA) are on high speed connections, but using thumbnails with large images is always a safer bet.

17. Clean, clear faces in images attract more eye fixations on homepages.
Comment: Humans are wired to recognize patterns and hard wired to other human faces.

18. Higher recall of facts, names, and places occurred when people were presented with that information in text format.
Comment: Good recall depends on the level of relevancy, good copy-writing and content usability (structure and display).

19. New, unfamiliar, conceptual information was more accurately recalled when participants received it in a multimedia graphic format.
Comment: It is known in the field of cognitive science that the more emotion involved in a learning transaction, the higher the retention and recall.

20. Story information about processes or procedures seemed to be comprehended well when presented using animation and text.
Comment: And the animation or text must be clear, easy to understand and in the language or conceptual world of the audience.

Types of Eye Tracking Technologies

1. Head-mounted tracker: Head mounted tracking devices as pictured at the start of this article (image from Poynter's earlier study) consist of a wire frame helmet that is mounted on the user's head in order to stabilize head and eye-movement.

2. Gaze-detection: This technique in addition to head mounted tracking has been around since the mid-1990's and featured as an interface device in virtual reality research. What's new with gaze detection are technological improvements.

In the Poynter III eye-track study, Stanford University derived  www.EyeTools.com used the new eye-gaze technology developed by Sweden's www.Tobii.se. In this system, the computer screen itself detects, captures and tracks the user's eye gaze patterns. Other vendors like Australia's www.SeeingMachines.com, Germany's www.Eye-Square.com or American www.EyeTracking.com offer headset and headset-free kits. 

Measuring more sophisticated variables...

I expect the next 15 years will see an increase in physiological measurements being used in consumer and usability research. Not because so called "traditional usability" techniques are inadequate, but rather because the field itself will help prove itself with the "hard proof" offered by the new technology. Already Eye-Square offers an additional skin-conductivity sensor to help detect such factors as shifts in sweat, temperature and heart-rate. Eye Tracking Inc. offers pupal diameter measurement as another way to gauge/track emotional response.

Further movements can been seen in  physiological research in Harvard University's lab run by Gerald Zaltman (author of the amazing book How Customers Think), where fMRI (functional magnetic resonance imaging) is being used to determine where data is being processed in specific regions of the brain. Now we're talking "hard proof". It's not what they see but what they think about what they see!

Problems reported with using eye-tracking for Usability Testing

Many eye-tracking firms (and the original companies that were founded) emerged from academic settings and moved toward selling commercial research services. The use of eye-tracking in usability research is fairly new and is recognized to lack empirical evidence regarding its effectiveness.

Eye-tracking technology changes every few years. However, most of the vendor websites do not offer detailed information about their specific technologies or approach (scientific basis, trials or double blind studies with their technology). The attitude seems to be, "eye-tracking is cool, so just do it"!

Take a quick look at the research literature on eye-tracking and a different story emerges.

Schnipke and Todd (2000) at George Mason University reported extensive problems in properly collecting eye tracking data, despite vendor training and one year's experience. They identified a host of obstacles such as ease of use of the system, calibration stability, pupil fluctuations and pupil condition quality as well as the issue of omitting users who wear glasses. The authors used a remote eye-gaze system.

Goldberg et.al (2002) at Stanford University and Oracle Corporation identified two styles of eye tracking studies: top-down (task oriented) and bottom-up (behavioral inferences). The researchers found that both styles of eye tracking studies must be adopted if eye-tracking is to become a routine usability methodology.

Pan et. al (2004) at Columbia University confirmed previous work by Rayner (1998), finding that individual characteristics of the viewer as well as the stimuli both contribute to viewer's eye movement behavior.

Eye-tracking seems to have a promising future. As the technology improves, so too will the research application, methods and action-ability of eye-tracking data. However, eye-tracking does not seem to be the holy grail of usability testing. The two biggest practical problems are calibration and complex reporting and analysis. In corporate usability settings, easy test set-up and quick design insight, guidance and recommendation are the most valuable elements of the usability research activity. If eye-tracking jeopardizes those elements then it looses some appeal.

Scrutinizing the quality and end results of new eye-tracking technology developments and methods will become a bigger problem as traditional academic spin-offs compete for eye-tracking services in a commercial capacity. As with any new technology it is important to remember why you are using it and what it can do for you. The Poynter research like many eye-tracking studies, provides another data point to validate or challenge your existing assumptions about user behavior. As Poynter's Howard Finberg put it, "Eyetrack III is a tool, not a solution".

The reality is that eye-tracking, while valuable, doesn't make usability testing any more powerful. It's what you do with the observations and the usability test data that counts.

Best Wishes,

Frank Spillers, MS

Electoral Ethnography

The British newspaper, The Guardian, is offering an interesting way for British readers to gain an insight into the electoral experience currently sweeping the United States. 

In a unique photography diary project Guardian Unlimited and Documentography have teamed up to capture the lives of American families in the run-up to the US election. Five photographers from the Documentography collective are living for a week with five very different families in California, Florida, Georgia, Ohio and Texas. Each day we will publish 10 photographs of each family and brief audio clips documenting their daily lives. The project runs from Tuesday October 26 until election day

The content feature, sponsored by Olympus, strongly resembles a primary activity of usability research: the ethnographic study. Ethnography is a technique developed largely by anthropologist Margaret Mead. It involves behavioral observation, contextual interviewing and analysis of users in their work, home or play spaces. The key strength to ethnography is context. Context provides insight into not only who users are (demographics) but what is important to them and what causes them to act and make decisions (psychographics). It's no surprise then that the Guardian employs the technique to offer rare, candid views of the American voter- especially this year in what is being dubbed the "most important election in our lifetimes".

Ethnography goes by a few different names. The most common include: Contextual Interviewing, Field Studies and Task Analysis.  My personal contribution to the field, called "Cognitive Archeology"(PDF) involves an analysis of user behavior and environment with an emphasis on understanding decision-making, problem-solving and the interaction of tasks, values and beliefs.

What is Ethnography?
Essentially, ethnography involves a "walk in their shoes" or a "day in the life" study. It is a method of observing human interactions in their social, physical and cognitive environments.

Leonard and Rayport note: "(Ethnography) is a relatively low-cost way to identify potentially critical customer needs. It's an important source of new product ideas, and it has the potential to redirect a company's technological capabilities toward entirely new businesses."

Dorothy Leonard and Jeffrey F. Rayport, co-authors of "Spark Innovation Through Empathic Design".

Core Ethnography Techniques

Contextual Interviewing: Visit users in the environment where they perform the tasks your website or application help solve. Why? Meeting users in a focus group room or conference room removes an important aspect of contextual inquiry- user behavior is triggered by environmental, social and emotional cues. Analysis of user tasks and goals can only be properly accomplished by going to user workplaces and conducting observation and open-ended interviews there.

Task Analysis: Observing users as they perform their "tasks" in the current state. This may included identifying coping strategies, shortcuts or gaining insight into the conditions, variables and contexts behind a user need or behavior.

Diary Studies (Pagers, Disposable Cameras): Giving users a diary to record their thoughts and experiences in conjunction with a pager and a disposable camera can elicit important information that might normally be overlooked in the presence of a researcher. These types of studies are valuable for gaining intimacy and probing matters of personal or emotional relevancy. This technique is best used in conjunction with contextual interviews, using the photos or entries as a way to trigger discussions, values or everyday routines that may normally be out of your user's awareness.

Some Data Analysis Methods

Affinity Diagrams: a method for brainstorming and determining relationship or "clustering" between information.

Task Flows: a method for organizing the flow of user actions, issues and system responses based on how users want the system to work (note: Use Cases are based on how the system will act- the subject of another post...)

Scenarios: a method for synthesizing observations in a scenario that provides meaningful insight into user intentions, expectations and actions with the new system.

Who would Margaret Mead Vote for?
The famous anthropologist Margaret Mead is credited with founding the area of ethnography within the field of Anthropology. In this year's election, Mead would probably vote for understanding the other side, its culture, what motivates it, what organizes its world (value and belief systems) and what causes it to act ;-)

On that note, here is an early offering from the upcoming Experience Dynamics research newsletter. Download the poster for your office cube. CubeArt: Do you practice these aspects of Margaret Mead's "user research"? (print it Landscape).

Best Wishes,
Frank Spillers

Design and Emotion

This is a late post relating to my paper presentation at Design and Emotion 2004 (conference in Ankara, Turkey). The entry contains some new writing cut from the paper due to size constraints.

Related paper: "Emotion as a Cognitive Artifact and the Implications for Products that are Perceived as Pleasurable" by Frank Spillers in the Published Works section on the Experience Dynamics website.

So, what does emotion have to do with design?

Emotion is one of the strongest differentiators in user experience namely because it triggers unconscious responses to a product, website, environment or interface. Our feelings strongly influence our perceptions and often frame how we think about or refer to our experiences at a later date.

When we think about emotion design and usability, we typically think of it as "keeping the user happy". This includes designing to minimize the common emotions related to poor usability such as frustration, annoyance, anger and confusion.

What emotions are you designing for?

Let's take a quick look at a few different design contexts and research that supports emotional design as we begin to understand how emotion is an essential design requirement.

1. On the web: A well organized website with a professional, "clean look and feel", with intuitive navigation and task-oriented functionality influences the following emotional reactions:

-Perception of credibility
-Trust
-Perception of security
-Overall perceived ease of use(PDF)

(See my previous post on how important graphic design is to a website)

2. For a software application: A software application that is task-centric, contains "just in time" features and performs robustly influences the following emotional reactions:

-Improved user satisfaction(PDF)
-Perceived software quality (user-perceived quality) PDF
-Subjective judgment or product "appealingness" (hedonic quality)

3. With a product or device: A product that carries aesthetic value and performs elegantly can influence the following emotional reactions:

-Deep attachment or bonding with the product (PDF)
-Perception of improved performance (attractive things work better)
-Perception of pleasure(PDF)

4. In a physical environment: Environments that facilitate the needs of social, physical and group dynamics with aesthetic considerations can have the following emotional reactions:

-Increased loyalty behavior
-Productivity increases

Is Emotion the Holy Grail?
Emotion is not an exclusive factor in defining a successful user experience.  Hekkert et. al. (2003) found that every product feature affects the experience, which can be complex and multi-faceted. Further, emotions are culturally specific and variable (Ratner, 2000), which may explain anomalies Desmet (2002) found in responses to the emotions of ‘desire’ and ‘disappointment’ when testing emotion in Japan, compared to responses found in the USA, Netherlands and Finland.

Intensity of emotional expression in product design is also highly dependent on the personal goals, attitudes and expectations the user brings to the product (Van Hout, 2004).

Time for a Paradigm Shift: Integrating Emotion and Cognition in Human Information Processing
Emotion plays a powerful role in our lives (Golman, 1997) and has gained significant attention as a priority area of study in interaction design (Jordan, 2002). Get Pat Jordan's book- previous link goes to Amazon- if you are interested in this topic or area of the field :-)

Traditionally, the field of Human Computer Interaction has distanced usability research from emotion. This practice is reflected by the field of Cognitive Science which, until recently, studied emotion as a separate, distinct facet of human cognition. Ratner (2000) noted that emotions and thinking seem so different, that we classify them as different kinds of phenomena:

“Emotions appear so antithetical to thinking that they are said to interfere with it. Clear thinking supposedly requires eliminating emotions”.

Cognitive research has tended to emphasize humans as problem solvers using the paradigm of a computer to describe human information processing. The notion that typical cognitive functions, such as thinking, rise above emotional processing, is a historic relic of the research practice of separating emotion from cognition. The current equivalent in usability research is the practice of separating aesthetics from interaction. For example, it is typical for usability practitioners to delay aesthetics until later and present “wire-frame” user interface specifications to clients without graphical treatment and to use “Latin” or “Greek” as placeholder content for prototypes.

Vygotsky (1962) and LeDoux (1996) believed that separating affect from cognition was a major weakness in the field of psychology and cognitive science. According to Davidson (2003), the perception that affect and cognition are independent, separate information processing systems is flawed. New breakthroughs in neuroscience using functional Magnetic Resonance Imaging (fMRI) have validated the assertions that cognition and emotion are a unified process. Gray et. al (2002) found that emotion and cognition ‘conjointly and equally contribute to the control of thought and behavior’. Not only does emotion contribute to the regulation of thought and behavior, but also cognition contributes to the regulation of emotion.

Contemporary views in Artificial Intelligence, are also embracing an integrated view of emotion and cognition. In discussing his book the Emotion Machine, Minsky (Steinert-Threlkeld, 2001) stated:

“Our traditional idea is that there is something called 'thinking' and that it is contaminated, modulated or affected by emotions. What I am saying is that emotions aren't separate".

What does all this mean?
For those practicing usability engineering, or responsible for ensuring a successful user experience, it means that you have probably trained yourself to focus on the critical, functional flaws of a design while underplaying the role of emotion. After all, usability is about the functional aspects of a design: Does it work?; Is it easy to use?; Is it easy to understand?; Is it intuitive?

Jakob Nielsen rarely celebrates the aesthetic value of design. On the other hand, his business partner Donald Norman, has transformed his tone to be less critical and more balanced in terms of recognizing emotional factors in design. In a reference to his article (2002) on his website (www.jnd.org), Norman stated:

“Up to recently, however, I could not make the connection between usability and aesthetics - they were distinct spheres of my life. Now, however, I have figured out the relationship”.

Keep the Emotional Data in your Design Decisions
Separating emotion from cognitive functions does not seem helpful from a research or design perspective. Instead, an integrated view of emotion and cognition appears to be taking hold, not only in neuroscience and artificial intelligence but also in interaction design corridors as well. I recently gave a small talk on the subject of Emotion Design at the Art Institute of Portland (Oregon, USA) where I teach a class on usability strategy, that was attended by 40 people from some of the largest corporations in the United States.

Take-away: 1) Create designs that look cool (aesthetic factor) and make your customers look cool (social factor) as well as usable,  2) Just because emotion does not seem quantifiable does not mean that it isn't (see research links in the first section of this article); 3)  Emotion design is a bottom line issue because, after all, customers make decisions based on feelings, perceptions, values and reflections that usual come from gut feelings (not logical, rational arguments).

Best Wishes,
Frank Spillers

Telematics Usability-Coming to a Car Near You

I recently returned from Detroit 2004 where I moderated several keynote expert panels on consumer needs, usability and the future of vehicle telematics. The conference is an annual event held in Detroit and attracts automotive companies, suppliers and analysts who are all focused on the bleeding edge of the "wired car" a la integrated communications, vehicle diagnostics, entertainment, real-time information and high-speed internet or satellite driven data "piped" to the vehicle.

What is telematics and what does it have to do with usability?
Telematics refers to in-car technology that helps improve the driving or passenger experience. This can include such things as automatic airbag deployment notification, vehicle tracking, personalized driving features, real-time traffic data, emergency aid, satellite radio or video, high speed internet and a variety of other vehicle monitoring systems, entertainment and "location based services".

Users interact with innovative features including Internet access, voice activation, and the ability to control audio and climate from the steering wheel, for instance, or with visual displays and/or buttons on the rear view mirror (as in GM’s Onstar).

Usability becomes important really fast in the automotive interiors environment. Distractions, errors and accidents can cause drivers to veer out of lane, collide or crash. The stakes are high, making driver distraction a core issue in the telematics industry.

If telematics usability is so crucial, why don't OEM auto manufacturers saturate the driving experience with intuitive and pleasurable interfaces? After four years of observing the ups and downs of the emerging Telematics industry with my company Experience Dynamics, I believe these to be the main obstacles:

1) Automotive manufacturing lifecycles surpass consumer trends. A key issue is how to deliver telematics solutions that do not outlive the automobile manufacturing lifespan. Several companies like Daimler Chrysler (U-Connect) and Nokia (Bluetooth) have turned to plug and play telematics that are automobile architecture independent.

2) Usability is not understood as a strategic business resource. The position of a human factors engineer (HFE) in the large automotive company is often technical and not strategic. For example, most of the work being done in the United States by the government and by industry/academia involves safety and driver distraction research.

HFE's have traditionally worked on the Human Machine Interface (HMI) inside the automotive cockpit to improve automotive interiors. Human Factors has always been seen, it seems, as an instrument, not a marketing and business aid. Worse, HFE's are often so "in the trenches" they do not know how to put the imperative telematics user experience into business terms.

3)  Driver behavior and emotion is the killer application. OEM's and Tier One's have been missing the point: Customer experience including behavior, social interaction and emotional response to and interaction with new telematics devices are key requirements for next generation telematics. Understanding the granular details of how customers currently behave with regard to navigation, problem-solving, group and individual communication, entertainment and other factors will define which companies use telematics to strengthen brand loyalty, while accelerating location based service adoption in the post-purchase period.

Consumers Adoption is a Critical issue in Telematics
Experiences with the telematics user experience are difficult for consumers to visualize, understand or imagine. In my keynote at the conference I told a story that resonated with many audience members:

A user once related his experience of telematics to me. He said that after purchasing a new vehicle he was given a survey regarding his preferences for telematics. After about twelve questions, the man ripped up the survey in frustration. He said he didn't know how to answer the questions: Did he want a screen display on the dashboard, where the radio sits, in between the seats or as a heads up display? He didn't know how he would like Mars, so to speak, because he had never been there yet.

A recent study by Frost and Sullivan echoes the finding:

"Basic navigation systems were given top ratings by respondents, with traffic information in particular being regarded as a value enhancer. However, a continuing lack of product knowledge appeared to be hampering the customers' ability to assess the benefits provided by advanced navigation systems".

Only in the last few years have we been exposed to navigation systems, advanced diagnostics and other early signs of telematics. Luxury car owners for six or seven years have been the guinea pigs for new telematics products. The mass market is slowly being introduced to telematics through rental cars, satellite radio and hands-free after-market kits.

Today, according to analysts Frost and Sullivan, 70% of vehicle fleets in Europe are aware of or use telematics, yet "More than half the respondents were uninformed about telematics' ability to improve customer relationship management (CRM) and substantially reduce the insurance premium" reports Telematics Valley.

Top of the Agenda: The Driving User Experience
Many Tier One suppliers that create automotive interiors, such as Johnson Controls, expressed concern for usability and user experience issues at the Telematics conference. This is a quantum leap from several years ago when telematics usability was secondary or absent in conversations. The real challenge now is for human factors telematics researchers to get out of the usability trenches and go for a drive (with the users). Only by transforming the tactical "safety and situation awareness" characteristics of human factors into the strategic "desirability and experiential-based" requirements will usability be able to produce results with a profound effect on business objectives and consumer adoption.

P.S. For more telematics issues, see the telematics usability articles and product reviews I have written for Telematics Update magazine.

Best Wishes,
FS

Graphic Design vs. Usability

The GUI Olympics (Graphical User Interface, pronounced "goo-ey") is an annual event where top graphic designers converge to design the latest and greatest "skins" for the Winamp media player, Windows themes and Icons.

The event is in its final week, and while it's wrapping up it might be useful to reflect on a couple of note-worthy items that relate to popular misconceptions of usability and graphic design...

The Logic of a "Skin"

The GUI Olympics are about designing interfaces that some graphic designers refer to as "skins". The idea comes from the habit of developing software and website code first and then adding the interface last ("the skin"). The concept is that many users prefer different interfaces, so provide a variety of skins and please more users!

There is nothing wrong with that concept as long as it is clear that it only works with highly specialized applications such as Winamp. It is even culturally expected in the Winamp community that skins will be created and made available.

When the philosophy of the 'skin interface' is applied to other applications, it becomes problematic. Here's why:

1. Interfaces in general are not "skinnable". A skin is an external dressing to some underlying functionality. The skin interface implies an "after the fact" make-over. Thinking about how users will interact with an interface at the end of the development lifecycle is too late.

Take-away: Interfaces represent the interactions, intentions, goals and tasks of your users. Greater success is gained from putting usability (architecture) before graphic design (dressing) early on in the process.

2. Skins are often dictated by visual appearance and not priority of user actions. Some skins look really cool. But since looking cool is the main objective of a skin, the underlying grouping of user actions, features or display of functionality is often lost.

Take-away: Looking cool for the sake of art is great if you can get away with it. However, in the business world where large scale websites or complex applications need to be intuitive and sensible, users want cool to be balanced with functional (usable).

What does the research say about Graphic Design?

Graphic design plays a significant role in the perception of trust of a website. You wouldn't know this from speaking to most usability professionals who spend their days arguing for usability not aesthetics. Worse, many usability consultants (you know who you are), personally do not favor aesthetics or beauty. I have worked with people who think that a table should be easy to sit at and still look like a park bench, as if a nice tablecloth and a big bouquet of fresh flowers didn't add anything to the seating experience...

1st Research Item: Website aesthetics influence feelings of trust (Karvonen 2000) PDF

2nd Research Item: Website aesthetics influence users' satisfaction, pleasure and perceived usability (Lavie and Tractinsky 2003) PDF

3rd Research Item: Website aesthetics influence perception of credibility (Fogg 2002) link see #6 in the tips list

Would you like a more usable "skin" with that user interface?
On the GUI Olympics site, if you search for a Winamp skin, it gives the choice to search filter for "Useable" interface skins. This seems hilarious to me, but might be a whole new business model for certain Operating System manufacturers. Imagine if you could buy the "usable" version of software or pay to have features or functionality removed from your current software packages? Hmmmm.

It seems GUI Olympics has created that filter since some of the skins go over the edge with graphic elements that it becomes difficult to find the play button! One interface I tested looked great to me, but when I started using it there was no progress meter (the bar you pull across to skip forward in a track). This is exactly the phenomenon I have found in my usability testing of hundreds of interfaces over the years:

Users will be attracted to great looking software, applications or websites with great excitement...but if the functionality does not act according to their expectations, feelings of abandonment or disatisfaction set in.

Bottom line: Functionality that does not deliver usability undermines aesthetic benefits.

Why and when users will favor cool over usable
The paradox, however, is that if users want to use the graphically pleasing (but un-usable) product due to some other stronger reason such as social influence or scarcity of content, they will. The Winamp skin interface is an example. Websites that have rare content but that are hard to use will still be used if the content is perceived to be exclusive to that site. This phenomenon was demonstrated by Lindgaard and Dudek (2002) who found that:

"Using a site with high appeal but very low in perceived usability yielded very high satisfaction, but low perceived usability scores, suggesting that what is `beautiful' need not also be perceived to be usable. The results suggest that web designers may need to pay attention to both visual appeal and usability".

Bridging the Usability and Graphic Design Gap

Usability engineers ought to live next door to graphic designers, not upstairs or downstairs from them. I believe that graphic design is extremely important, if applied with a sensitivity to usability. Typically usability engineers will hand "wireframe" prototype designs to graphic designers, who often will alter the information architecture in order to enhance the look and feel.

Usability professionals need to wake up to the fact that aesthetics are not a trivial afterthought, but an integral part of the user experience. We need to understand the value of graphic design and visual designers need to understand the value of usability. I had to learn this the hard way. In a previous job as a User Centered Design specialist, we spent $30,000 perfecting the usability, interaction design and functionality of a new website with several rounds of usability testing. Our client handed our user interface design specification over to an agency (who now mysteriously claims to be a user centered design agency). The agency added their graphic design to the website and completely changed the navigation system in order to add a logo and a branding element.

The lesson: Graphic Design can "hijack" usability efforts if the graphic design team is not "on board" with usability. This is probably why these days more and more graphic artists (like the students at the Art Institute of Portland where I am currently teaching a class) are learning about usability and have a sensitivity for its user-centered intentions. Here's an interesting paper that discusses balancing usability and graphic design on Yahoo!

I believe that closer collaboration and more respect between usability and graphic design is inevitable, as is the overlap between usability and marketing. More on that in another post...

Best Wishes,
FS

How Usable is Jakob Nielsen?

In an earlier posting, I said: "Jakob Nielsen has an unhealthy monopoly on Usability Consciousness. He promotes best practices, he preaches obedience to his guidelines and when he postulates opinions they are interpreted as instructions".

So what's the big deal? Jakob is an internationally recognized usability "celebrity". His books sell many copies and his website gets lots and lots of traffic.

Who is Jakob Nielsen?

Jakob Nielsen has become a hot item over the last five years with regard to website usability. Jakob Nielsen has a computer science PhD and is Danish by origin. See his bio for more... The Danish part is significant I think, because some of his prophesies get lost in translation.

For instance, Nielsen said:

"In the future, first of all, websites will be designed by my guidelines ... for the simple reason that if they don't, they are dead." - Jakob Nielsen.

Some colleagues of mine at a former company said "Look at that, Nielsen says if you don't do what he says you're dead [meat] (American slang for a threat)". If you re-read the quote, I believe the implication Nielsen was making was "you will disappear- become extinct" not "if you don't listen to me you are toast".

So what's the problem with Jakob Nielsen?

Jakob Nielsen is still very popular outside the usability community. Amongst his colleagues however, his popularity has been eroding steadily. Why? There are a couple of reasons for this.

1) Guru role. Nielsen plays up the guru role and gloats in the press coverage of being called a "king" a "guru" and "almost as big as Elvis". Gurus seem to fall into two categories: the silent and humble type  or the boisterous and arrogant type. Nielsen has been perceived by many as falling into the later category. However, I think the real gurus are his public relations firm, the Antenna Group, who are responsible for his glamorous international media coverage.

2) Professional ethics. A few years ago (circa 2000), Nielsen posted on his home page that his fees were roughly $35,000 for a half day high level strategy consulting session. This is where the credibility started to erode for me. If anybody thought usability was expensive, Nielsen certainly contributed to that perception (despite his current claims that usability is cheap: "Usability for $200" and "ROI for Usability").

Around the same time, Nielsen failed to disclose his personal conflicts of interest in several articles.  Before 2000, he always disclosed where he had a conflict of interest (i.e. serving on the Board of Directors) with regard to pitching a vendor product or service in his column. For example, he wrote an article advocating use of Google Adwords and didn't mention he was on the Technical Advisory Board. After doing that several times and being criticized by his peers, he came clean and apologized for not disclosing his conflicts of interest.

3) Useit.com. Nielsen's personal website defiantly retains its 1997 look and feel. Nielsen's justification for his site seems ridiculous (he says he is "not a visual designer and didn't want to spend money to hire an artist"). Reading between the lines, like many usability engineers, Nielsen does not seem to personally value aesthetics and that shows in his work. His site is also full of double-standards. A check of dead links on his site with the free tool he recommends (Xenu) in his article "Fighting Linkrot" produced approximately 458 dead links (including images, links, spacers and outgoing links).

Bitching like Elvis

One of the things I have noticed about people who take Nielsen's teachings at face value is that they end up communicating like him. The blaming, critical and self-righteous tones that characterize Nielsen's articles and interviews are not to be confused with how a professional usability consultant ought to communicate. Of the hundreds of people I have trained in the past few years, I have noticed the "Critical Jakob" in their findings. The danger is that armed with Jakob's influence, we can assume that we have a hammer large enough to break anything. A reader comment to Jakob's partner Tog a few years ago raised this issue. Tog's response was that this communication style was for marketing purposes only and that it was not the way he and his colleagues speak to their clients.

It's rich to criticize something as if no other dependencies exist. Even better to pretend like you have all the answers and that no humans (with feelings) were ever involved in the design. By taking the National Enquirer (a supermarket tabloid newspaper) approach to communication, Nielsen is doing a disservice to the usability practitioner community by not imparting best practices in communicating usability insights. Is it asking too much to have usable communication?

Jakob Nielsen has done a good job of inflaming the people he is supposed to be enlightening. Some examples of the things Nielsen has said that have inflamed people include:

* "Flash is 99% bad"
* "These companies are narrow-mindedly insular, and populated with lifers.." (referring to companies in industries who don't get usability)
* “Here are just few examples of the BMW 745i’s clueless interaction design” (See the Open Letter to Jakob Nielsen this one provoked)
* "PDF: Unfit for Human Consumption" See the rebutal from Adobe ePaper staffer...

The Backlash to Jakob Nielsen

While the tabloid marketing approach might get read attention, it also produces its share of criticism. There have been many attacks on Jakob Nielsen, some of them humorous, hysterical and some more serious.

1. Example from Marketing Profs, Jim Kukral "Being Jakob Nielsen"(member login required)

Why wouldn’t the King of Usability want to tackle the greatest challenge of them all: Designing a beautiful AND usable Web site?

Isn’t that what all Web professionals should strive to do? Why does there have to be a barbed-wire fence between designing a usable site and designing a graphically pleasing site?

There doesn’t, and to Mr. Nielsen’s credit, he doesn’t believe there has to be, either, even though his Web site shows me otherwise.

So what’s the hold up, Jakob?

2004 is upon us. Perhaps you have a Web site redesign in the works. Or maybe you’re finally going to build that company intranet you’ve been dreaming of.

Are you going to spend the money for a good designer and usability person, or go the way of Jakob Nielson and forgo the design part?

Obviously, your budget may have something to say about spending extra money on a designer, but don’t worry about it.

Just say, “Jakob Nielsen doesn’t feel it’s necessary, why should we?”

2. Example from Usability News, George Olsen "Response: The Backlash against Jakob Nielsen and What it Teaches Us"

Nielsen has had a bad habit of presenting personal opinions as research fact. The latest example is his rather bizarre claim 90% of (his proprietary and not-disclosed) usability guidelines will likely be achieved by 2017 (see Improving Usability Guideline Compliance).

A personal thorn in my side his been his insistence that blue is - and will forever be - the only appropriate colour for links. Now I have a background in graphic design and I know numerous ways to make clear something's a link. No, I don't have academic research to prove this, but I've got many a successful site. But yet, I still have to deal with business decision-makers who believe Nielsen has "proved" this point.

3. Example from Clay Shirky's "An Open Letter to Jakob Nielsen"

(Nielsen suggests enforcing his usability guidelines on the web as a way to improve usability)

Let me quickly address the least interesting objection to your idea: it is unworkable. Your plan requires both centralization and force of a sort it is impossible to achieve on the Internet. You say

  "...to ensure interaction consistency across all sites it will be
  necessary to promote a single set of design conventions."

and

  "...the main problem lies in getting Web sites to actually obey any usability rules."

but you never address who you are proposing to put in the driver's seat - "it will be necessary" for whom? "[T]he main problem" is a problem for whom? Not for me - I am relieved that there is no authority who can make web site designers "obey" anything other httpd header validity.

Breaking the Monopoly on Usability Consciousness

Usability is about understanding human behavior. For one person to dictate how customers behave and how a field should apply best practices seems counter-productive. There are many wonderful "thought leaders" in the Usability and Human Computer Interaction world that many outside of the practitioner community rarely hear about. Jakob's colleague Donald Norman is just as influential, if not more, but his contributions seem more palatable perhaps because they help you learn instead of making you feel inferior. Or people like Brenda Laurel (a pioneer in User Interface Design and 3D interaction) who for a short stint was with Nielsen-Norman group. And there are many more who have contributed to the field...

Nielsen's famous "heuristics" (guidelines for web usability) are another area that shows that his popularity is more public relations than credit of invention. For instance, few people realize that Nielsen co-authored the original publication on heuristics with Danish colleauge Ralph Molich and then later with Robert Mack and others. Yet you rarely hear usability folks saying "Nielsen-Molich heuristics", it's "Jakob Nielsen's heuristics".

Contributions to developments in the field are compromised if practitioners must tip-toe around the shadow of what Jakob Nielsen said. For example, Nielsen is very vocal about 3D and virtual reality interfaces. He claims that evolution did not intend humans to navigate in 3D space. Since I did my masters research in the usability of virtual environments, it appears to me that Nielsen has never looked at any of the research in 3D usability that has come out of British universities or the Human Interface Technology lab at the University of Washington, for example.

Reforming with Research Based Usability Guidelines

What we need is a reform of how usability is "trickled down" to the masses. As awareness grows to commercial bias or self-serving interests in usability research, we will see a greater emergence of independent research-based usability guidelines such as these from the National Insitute of Health.

Follow up: Read Jakob's Ladder for a very comprehensive profile of Jakob Nielsen.

Best Wishes,
FS

Progressive Disclosure- the best interaction design technique?

What is "progressive disclosure"?

Progressive disclosure is an interaction design technique that sequences information and actions across several screens in order to reduce feelings of overwhelm for the user. By disclosing information progressively, you reveal only the essentials and help the user manage the complexity of feature-rich sites or applications. Progressive disclosure follows the typical notion of moving from "abstract to specific"; only it may mean sequencing interactions and not necessarily level of detail (information). In other words, progressive disclosure is not just about displaying abstract then specific information, but rather about 'ramping up' the user from simple to more complex actions.

In its most formal definition, progressive disclosure means "to move complex and less frequently used options out of the main user interface and into secondary screens".

A few familiar examples of progressive disclosure

An example of progressive disclosure is an online news article that is spread across four screens (with a Next Page link at the bottom). This use of progressive disclosure serves advertising objectives (showing banners on each page) and not the user's task. Another example would be a site that explains a product by making the user click through 4-5 pages of overview/benefits information before revealing the price of the product. The idea here is that if the user reads the product information, they will accept the price more easily. The problem with that approach is that it does not accommodate free-form exploration, a typical behavior on the web.

In its purest format, progressive disclosure is about offering a good teaser. A good teaser can include the following:

* A sample of what is next
* An introductory task that is most common
* A high level view of what is expected
* A wizard that walks the user through the task
* A button that leads to more advanced functions (such as editing)

Progressive disclosure says: "Make more information available within reach, but don't overwhelm the user with all the features and possibilities".

Short History of Progressive Disclosure

Progressive disclosure is a concept that has been around since at least the early 1980's.  The technique caught the attention of user interface specialists with Jack Carroll's lab work at IBM (1983), where he found that hiding advanced functionality early on led to an increased success of its use later on. The approach dubbed "training wheels" is one of the only references validating the technique. Carroll and Rosson (1997) pointed out that no empirical evidence exists regarding the effectiveness of progressive disclosure and that the training wheels approach only studied a "single computer application (word processor) and a single interface style (menu based control)". While independent usability studies and consultancy research (including our own) has shown that appropriate usage of the technique is valuable, more empirical research is clearly required.

The software vs. web design environment

Historically, progressive disclosure is a concept that came from the software usability experience. It is clearly easier to apply to software than it is on the web, which is probably why you don't hear all that much about how to do it effectively on the web. In software, the interaction is between dialogues and 'fixed state' interactions. On the web, interactions are chaotic, randomized and dynamic due to the fact that hypertext is a non-linear media.

In the software world the audience is predictable and targeted, making learning styles more predictable. On the web, it's anybody's guess who might be using the site. The website visitor might be a particle physicist, a teen or a grandma. Learning styles, comfort levels and expectations differ greatly. This is perhaps why you hear a lot of references to progressive disclosure in conversations and interviews, but rarely any ideas about how to apply it effectively.

Usability guru Jakob Nielsen mentions progressive disclosure regularly, but has never explored the subject in great length to my knowledge. Nielsen has stated:

"Good usability includes ideas like progressive disclosure where you show a small number of features to the less experienced user to lower the hurdle of getting started and yet have a larger number of features available for the expert to call up". Sitepoint interview with Jakob Nielsen

"Progressive disclosure is the best tool so far: show people the basics first, and once they understand that, allow them to get to the expert features. But don't show everything all at once or you will only confuse people and they will waste endless time messing with features that they don't need yet". Slashdot interview with Jakob Nielsen

There are dozens of references on the web (at least the first 7 pages of a Google search) of people saying progressive disclosure is valuable and recommended. Guidelines include it as an error control device and colleagues mention it often in corporate design rooms I personally frequent. The common notion is that it's good, everyone knows it, so just do it! I have yet to see anyone provide evidence of how to use it effectively on the web...

Using progressive disclosure effectively on the web

The marketing approach to progressive disclosure is to get excited about the features and force users to partake of the excitement by making them wade through them all. There's only one problem with that: if you want to get someone excited, creating a feeling of overwhelm does not strike me as a good way to get someone excited. Instead you want to roll a small snowball down a hill and hope it gets bigger and bigger leading to an avalanche! This is the goal of progressive disclosure from a marketing standpoint.

The best way to think about progressive disclosure on the web is: "Only show information that is relevant to the task the user wants to focus on, on any given page". Context sensitive advertising has finally figured this one out with regard to the text ads that Google pioneered.

Progressive disclosure is an interaction design technique that emerges out of the insights gained during Task Analysis (user observation of tasks). Observing users in the field, allows you to understand their workflow outside of your technology. This insight gives you the data you need to prioritize and sequence content functionality.

The main thing to remember about progressive disclosure is that it you will be able to use it correctly if you have conducted task analysis (behavioral observation) with your user base. Observing users in their native problem solving environment gives you the insight about how they interact with the information. By observing someone's eating habits, you'll know whether they typically look at the desert and drinks menu at the start, in the middle or at the end of the meal. You'll discover whether they like to eat the main course or their salad first and whether they drink before a meal or at the end of a meal.

Examples of progressive disclosure on the web:
* Learn more link
* Related topics link
* Overview of account information on the first screen
* View more details link
* Advanced search link

Why is progressive disclosure the most important interaction technique?

Progressive disclosure is powerful because it embraces several good design principles:
* Advocate for users with different needs (experienced and non-experienced users)
* Limit what you show on a screen
* Give access to the low hanging fruit and de-emphasize infrequent tasks
* Only show users what they need when they need it
* Focus the interface on making the user successful at the start

Benefits of Progressive Disclosure:
* Remove the need for the user to explore and examine the interface first
* Allow the user to chunk the task in a sequence that matches their expectation
* Reduce cognitive overload
* Increase the efficiency and ease of use of the site

Dangers of Progressive Disclosure:
* Users are forced to wait until you are ready to show them
* Repeat use may not require progressive disclosure (depends on the task)
* Over-constraining what users see (or dumbing it down too much)
* Assuming you understand what is the most popular, common or important task

Progressive disclosure has a good name as an interaction design technique and for good reason. The downside to it, is in its inappropriate usage. Microsoft Word, for example, is full of many inappropriate uses of progressive disclosure, such as the auto-hiding extra menus that you have to repeatedly activate even if you don't use the menu item (the down arrows that appear at the bottom of any menu).

Progressive disclosure can be powerful. It allows users to orient to a screen, figure out what they need to do, and do it in steps that reveal more complex information as they go along. As a design technique its fallacies can be prevented by basing the progressively disclosed tasks and information on actual observed user behavior. If you can realize that the technique is an outcome of data gathered in task analysis and not just a "design idea" you will get more mileage, more accuracy and you will be more successful using it as a tool.

Best Wishes,
FS

Quote this as: (2003) Spillers, Frank. Web Usability Best Practice Handbook. Experience Dynamics.

Methodology or Mythology?

Methodology Madness: or "caveat emptor" (buyer beware)

What you buy or "buy into" influences how you think about something and how you represent that information in your mind is what cognitive scientists refer to as an "internal representation". Whether you buy usability services or not, at some point along the way I am sure you will or have encountered "methodology madness", and maybe you don't even know it.

What is "methodology madness"?
Methodology madness in the usability services and products area refers to the espousing of convenient beliefs, "truths" and proclamations about the right way or new way to do things. The methodology is typically proprietary or masked and is typically part of some form of a sales pitch- either for a report or a "customer experience management" solution. The "right or new way" implies that the approach is more refined, more advanced or a best practice.

Methodology madness is not new to usability consulting; in fact it exists in many industries. In terms of the usability industry, the problem with proprietary methodologies is that they are often inaccurate or distorted versions of the truth. The other obvious problem is that proprietary usability methodology, techniques, or research serves that company's interests, with a clear commercial bias.

A fair degree of usability nonsense seems to be emerging as the industry grows, and it's main motive is sales and competitive differentiation. Further it is hard to tell what is nonsense and what is valid, witnessed by the fact that I have met many usability consultants who believed certain methodology myths. Like many of my colleagues, I have even fallen for some of the mythology because it sounds so convincing.

Let's face it: it's hard to think critically about something when it's packaged in a compelling way and important details are withheld in the name of confidentiality.

To help identify the madness, let's look at just a few common methodology myths still in circulation today:

Claim 1: "Usability testing must be conducted in the user's natural setting".

Source: This one comes from a leading provider of a semi-proprietary online customer experience solution that uses panels of users in their homes, traffic log data and analysts to generate reports.

Problem: There is no evidence of this claim in the Human Computer Interaction literature (the field where usability comes from). While the claim makes sense, it dissolves when you trace it to the usability technique where it was borrowed from: field studies. In field studies (aka Task Analysis, Ethnographic studies, contextual interviews) it is absolutely essential that the user's environment be observed and assessed. The point in field studies is to note interactions and influences of the environment. In usability testing, the point is to gauge if the website or software works to expectations. This has very little to do with the user's setting, or PC settings etc.

Claim 2:"You need to test your website with hundreds of users".

Source: Same as above.

Problem: This belief sells statistics, not usability insights. Since the majority of people are most comfortable with statistical data (quantitative), this claim again sounds convincing. The flaw however, is that usability testing is a qualitative research technique (observation is the metric not numbers). In qualitative research the research rules are different and it is normal to have small sample sizes e.g. 15-40 users. Usability testing is about observing actual user behavior and capturing expectations. Insight is the indicator, not statistical significance.

Claim 3: "If it takes more than three clicks, forget it".

Source: Unknown. Probably went around dozens of startups in Silicon Valley in the late Roaring 90's.

Problem: This "3 click rule" metric is e-commerce centric. Three clicks to the user destination is a metaphor for saying "don't take the user down the garden path to do something". The 3-click rule losses validity in other domains where users will naturally click 10 times to research an issue or purchase.

Claim 4: "Navigation is not important. Users don't care where they are in the website".

Source: A popular "customer experience" guru and evangelist.

Problem: This is a new one (Feb 16th 04) cycled back from something guru Jakob Nielsen said a few years back with the effect that navigation was "overdone" on many sites. In the new version, we are told "consistency is NOT necessary", and does not apply to websites. Outside of falling down on the floor with laughter, the problem here is that while users don't appear to be consciously concerned with navigation, their unconscious behavior indicates otherwise. A simple fact that every seasoned usability practitioner knows is that consistency increases ease of use (whatever the medium). Again, the prescription references the insights to "listening labs" (a reframed usability testing lab with questionable methodology of it's own).

Skilled observation by professionals that understand consumer cognition can go a long way to prevent sweeping generalizations about user behavior. For more on the topic of understanding unconscious customer behavior, see Gerald Zaltman's new book How Customer Think, where research shows physiological evidence of consumer behavior using brain scans.

Claim 5: "Website usability can be measured by proprietary software, agents or algorithms".

Source: a) a now defunct company and b) a new consultancy with a similar story.

Problem: Because usability involves the understanding of complex, dynamic, state dependent cognition it is virtually impossible to model user behavior with a bot, agent or algorithm. For example how can a machine model semantic interpretation? Machines can't.  I worked for a time with the company who claimed the had invented a "technique that models human perceptual, cognitive, and motor behavior, and is programmed with a set of characteristics and Web-browsing behavior that represents the way an average user sees, thinks, and moves through a Web site".

The claim is completely false and was disproved by a world authority at Xerox PARC. I even compared four automated tests to four equal real usability test and had consistent dramatic failure of results from the automated algorithm approach. I realized that it is impossible to model how a user makes sense of a website, how they interpret content and to predict their expectations and train of thought. Yet, a new usability consultancy (that refuses to provide basic details about their methodology) has "invented" a proprietary algorithm for assessing competitive usability performance capturing data such as scrolling, scanning, typing in data, reading text, clicking and annoyance. Sounds too good to be true. You don't get to find out unless you become their client the President told me...

What is the anti-dote to methodology madness? As the Latin term "caveat emptor" implies (let the buyer beware), the best thing you can do is think for yourself, do your homework, compare and contrast the information. Ask a seasoned practitioner if you are not sure.

For the usability consulting industry, the agenda ought to include the following:

1) Clarifying and providing rigorous detail about proprietary methodologies (including peer reviewing).

2) Promoting integrity by serving prospects and clients with non-biased and non-partisan information.

3) Building and expanding upon existing agenda-free techniques and methods that service the greater good of the community.

I personally don't believe that "new" usability methodologies should be kept proprietary under the auspices of commercial protocols. Best practice research is not like a new technology or invention. Usability is about understanding user behavior and there is nothing proprietary about human behavior.

I also don't think it serves the industry or the pursuit of integrity for that matter, to claim that a technique is the "secret sauce". That's like one lawyer saying they have a better methodology to practice law then another attorney. There are people and companies who are competent and skilled and there are those who are not.

Best Wishes,
FS

 

Are you hiring what you think you are hiring?

Part 1:Clarifying skill sets

This User Interface (UI) Designer position correctly states that UI designers create navigation maps, functional specifications and design requirement documents. Hint: They don’t code anything.

Yet a recent polling of my colleagues and clients shows that the common usage of “User Interface” person means someone who does "front end" (interface design) and the "back end" (programming). There are many UI Designers out there who design the usability and they have a background in the area. However, because programming is being conducted simultaneously, many people fall into the “split-mind problem”. It is hard to focus on the user's needs while thinking about the system's needs.

Years ago, before working exclusively in usability, I programmed in C++, C, HTML, Java and some virtual reality and artificial intelligence languages. I can verify that there is a huge mental leap between the "front end" (interface design) and the "back end" (programming). It is extremely difficult to do good usability if your focus is code.

Takeaway: Be careful to clarify your conversations with an agreed definition of UI Design skills. In the usability community, it means the above (see ad). In the programming community it means the job involves code. There is a difference. Make sure you know what you are hiring.

Part 2: Focusing on strategic insights

Fact: Usability is not Quality Assurance (QA) and can not be performed professionally by a Quality Assurance specialist. Would you have your plumber take care of legal problems?

People who hire “Usability Testers” and QA specialists to handle usability requirements, seem to think usability is purely functional. They, like the authors of the above (second job ad), contribute to a one-sided view of usability, namely that usability is purely a tactical step in good software development.

Usability is actually highly strategic. Design changes can impact: markets pursued, how audience segments are defined, whether marketing and business goals are altered, whether resources need to be allocated to change or add new functionality etc. Compare this to the outcome of QA where fixing a button, moving a link or adding an image might be the result.

Takeaway:If your usability requirements are going to be covered by QA, you may be missing the forest for the trees. Never substitute QA for usability in your web development lifecycle.

Best Wishes,
FS

The Future is Usable: New Communication devices

Consumers can feel poor usability with great intensity on a cell phone or with a new data enabled device-- unlike a website, software application or operating system that involves more relaxed cognitive states (sitting down and learning the system). When you need to make a call, send a text message or web browse on a handset it becomes obvious what ease of use is and isn't. A large part of the design problem in mobile user experience is user environment (mental, physical and social). Think about the difference between sitting down with a cup of coffee watching windows pop up, pages move around and your ability to close windows, back up with the browser etc. Now think of holding a communication device and interacting with 1.5 inch  screens while standing up, walking around or in the social presence of others.

A recent survey by MIT found that adults in the US rated cellphones the top of the list in technologies they hated but couldn't live without:

"An annual Massachusetts Institute of Technology survey, known as the Lemelson-MIT Invention Index, found that among adults asked what invention they hate most but can't live without, 30 percent said the cell phone.

Alarm clocks were a close second, with 25 percent, followed by the television with 23 percent and razors with 14 percent. Microwave ovens, computers and answering machines also earned spots as detested technology".

John Quain at e-week believes the popularity problems wireless technologies have been having in the US in the past few years (compared to Europe and Japan for instance) have to do with desirability.

"Part of the reason again is the added expense in a competitive market, but the main reason is simply that consumers don't think they need it".

The need to target actual user behavior in design requirements. Field testing is essential for devices that are deeply embedded in the users physical and social context. New devices require vigorous and frequent usability tests and fluid user interface specifications that can morph to changes based on new user needs that emerge during your usability test(s). This is especially true for localization of services when introducing new products into new markets. 

Mobile devices also need to support "user-compatible" configuration.Mitch Kapor (founder of Lotus Notes) explained his frustration with hitting a dead-end when trying to configure the email settings on his new Nokia 3650 cell phone. His story serves as an important reminder of the need to make settings configuration "user- compatible". More on Configuration Usability later...

Best Wishes,
FS

 

Ad conversion rate influenced by time (not click rate)

Time is an important design variable to understand. Your user experience is effected by it no matter what user experience you are serving up and the rules are different for every context. For example, the "three click rule" (users must get to their destination within three clicks) applies to e-commerce primarily but not to mortgage education, financial services usability or reading the New York Times online.

Time Usability 101: Design for asynchronous interaction. Asynchronous means users don't do it all in one setting. Environmental distraction is inherent to any user experience: the pasta is burning, the kids are screaming, the documents are being tracked down, printer cartridge is being changed, someone interrupts...

Here is some fascinating research about conversion rates and the influence of time:

Up to 85 percent of conversions occur after the day the most recent advertising impression was served, according to new research from Advertising.com. The results seem to show online ads can have an impact on buyers' purchasing decisions long after they view an ad.

For the five-day monitoring period, approximately one-third of all conversions happened on the same day that the impression was served, but only nine to 11 percent occurred within three hours of the impression being served. The longer periods of monitoring revealed that up to 85 percent of conversions occurred days after a user was served an impression.

"These findings show what a significant portion of conversion activity takes place well beyond the impression," said Scott Ferber, chief executive officer of Advertising.com. He said that the data shows that metrics such as per-click or per-action may not give marketers an accurate picture of campaign results. "It also reveals how lasting the effect of an impression can be," he said.

This data supports the theory that users' consumption of the Internet is often patterned and repeated on a daily basis.

If you know when your audience is going to be online or converting, there is some value to targeting by time of day."

The implications for web analytics analysis, advertising and content management are interesting:

1) Ad impressions go through a 48 hour or so window of decision making activity. Decision making being influenced by time and potentially access to external information, persons or environments.

2) Interpreting user behavior through a web analytics perspective ought to include a sensitivity for the impact of time (i.e. conversion rates are non-local meaning they aren't isolated to immediate on-page actions as is the current view of traffic log analysis).

3) Understand how you are supporting time usability. Do you offer good time-out messages, good error protection from browser instability (if serving a web based application)? Does your content support good decision making across time?

4) Consider your website usability strategy as a variety of ad impression. Already, independent studies show that website usability (ease of use) increases the likelihood of purchase. More on this later.

5) Get ahead of the competition and synchronize ad impressions with conversion behavior. In other words, throw out the purchase cycle across time and serve up product and service content that parallels return visitor activity. In some cases with high ticket software solutions, getting clear about this may lead to strategic leaps in revenue generation that otherwise was pegged as just low conversion.

If you'd like to check out the science of time usability, have a skim of this paper (PDF 196K) Daniel Loewus-Deitch and I (Frank Spillers) published at the workshop on the Temporal Aspects of Tasks in September at HCI 2003 in Bath, England.

Best Wishes,
FS

Patenting usability improvements- going over the top?

Summary: Questioning Jakob Nielsen’s recent advice about competitor data and his advice to patent usability best practices.

Slashdotters react to Jakob Nielsen’s recent column. And they raise some good points:

• "The reason this logic doesn't follow for many software engineers and software patents, is that the stuff that gets patented is sometimes simple, and a horrible waste of cycles and time happens globally as people work around the patents".

• "Usability is required on *all* sites, not just the ones that have the resources to patent everything".

• "So if I come up with a great usability enhancement, I should patent it? How does that increase the usability of the web overall? We live in a sea of unusable web sites and horribly designed programs.. now he's saying "hey, the goal is actually not to make web sites more usable. The goal is to come up with usability enhancements that one or two web sites will use. The other sites can go stuff themselves."

• "...is just going to be a directory of patents? Just swap out any point where there is actual usability advice with the relevant patent number and you're done".

The key issue here, I think, is that Jakob Nielsen has an unhealthy monopoly on Usability Consciousness. He promotes best practices, he preaches obedience to his guidelines and when he postulates opinions they are interpreted as instructions.

More later on how this has eroded Jakob Nielsen’s popularity within the usability community.

Best Wishes,
FS

Email Marketing- New Legislation

In case you missed the big news in the last few weeks, there is a new anti-spam law that was enacted on January 1st 2004. The rules for sending email have changed. Marketing Sherpa has a good summary:

Update Memo: CAN-SPAM Good News & Bad,
Bad, Bad, Bad News

SUMMARY: We just spent two and a half hours quizzing the FTC's attorney on exactly how the new law affects emailers. Here's a quick, handy round-up for you:

Good news: It's fairly easy to comply with message content.

Bad News #1: It's hard to comply with opt-out rules. Welcome to suppression file hell...

Bad News #2: The law applies to more types of organizations and email than you think.

Bad News #3: There's no "grace period."

Bad News #4: And, almost anyone can sue you.

Read on... and then forward to your legal department:
http://www.marketingsherpa.com/sample.cfm?contentID=2576
(Open access = permanent)

I don't know about you, but my spam hasn't reduced since the start of the year :-(

Best Wishes,
FS

Site Search can be flattened by Usability

If you're using a search solution on your website (Ask Jeeves, EasyAsk, Google, Inktomi, Mercado, Verity etc) or simply offer a search interface that users can find products, information or documents with-- you may be missing the main feature that will make it work (and give you the return on investment you want): Search Interface and Search Results Usability.

The search interface runs into these typical problems that most people don't consider when shopping for a search solution:

1) Search Interface: how the database works typically defines how the search interface will look (see the "Boolean logic" and "Natural Language" issues below)

2) Search Results: how the user configures the search is going to determine how great the results are. Also how the search database works will determine how *robust* search results can be displayed. Displaying search results in a readable, understandable and helpful manner is instrumental to designing the user experience effectively.

Search appears to be perceived as a highly valuable resource by users and marketing managers as an essential "help" feature on a website. As many of my clients can attest to, finding a good search vendor is a bit of a headache. One of the big issues is the feature many search vendors sell called "Natural Language Search".

Question: What is Natural Language Search?

Answer: Natural Language Search (NLS) is a more robust version than standard "Boolean" searching. Boolean logic says "And this, Or that, Not that" such as with this search interface http://search.internet.com/ (Probably the worst search interface on the web?) In that example the interface exposes the *needs of the database* e.g. you must enter this and that but not that.... the problem with *exposing Boolean logic* parameters to the user is that if they get it wrong, they may end up with nothing or something off-topic.

NLS really is a back-end feature. NLS database matches "fuzzy" records in the database so users can type "What's for lunch?" and get intelligent results back (supposedly). Ask Jeeves is the classic example (though it is not that great to put it mildly). Google of course has perfected it and Amazon too...but those are very expensive databases used by those two companies.

The main usability question to ask when reviewing a search vendor or evaluating the user experience of your own search solution is, "Can you customize the results?" and "Can you simplify the search interface?". Those two things matter for ease of use and value to the user. Search Results are as important as the interface.

I noticed Amazon is "culling" search results more...so that if you search "Interaction Design" on Amazon they cut it to 2 top results and then "View 230 others". The search rule system is imposing something onto results to prioritize them by popularity (or other variables).

Question: Why is it that when I want to find a Starbucks in an unfamiliar place, Starbucks store locator search is useless?

Answer: Starbucks coffee shops occupy multiple addresses in the same area of town across the country. When a colleague you meet wants to have coffee and says: "Meet me at 1134 NW Glisan (Portland, OR) Starbucks" (more specific then meet me at the one Downtown on 3rd for example), when you type in the address it gives you: two in the area, and then, a list with details for 163 stores in the entire city. Even when reducing the default 5 mile radius to two miles, it acts the same way. The Glisan store is on one of the 20 pages of search results! (try it for yourself)

What is happening I think is that the database is not matching inputted data to the store details page but instead is serving up all results based on geographic data. Result: annoying and unhelpful. Possible solutions include filtering results to match address to actual store pages; adding semantic data to the result (e.g. "across from the Library"); allowing users to search with visual maps of the city and filtering stores based on simple user selection (as in the example below). Starbucks unhelpful search has caused me: to go to the wrong store; pick up the phone and call to confirm; trade 3-4 additional emails to confirm (with the business person I am usually just meeting for the first time). read: hasstle

Here's an example of a successful search implementation:
My client used a Boolean search interface before with a 94% search failure rate during usability testing, we changed it to this one, leading to a 44% increase in registrations, two months after the change in search interface and search results usability.

Best Wishes,
FS

Quote this as: (2003) Spillers, Frank. Web Usability Best Practice Handbook. Experience Dynamics.