Category: User Testing

I propose a new job description called “Real Life User Experience Testing”, whose role is to assist  software and website developers by luring them away from the white board and drop them into reality.

I think about real life user experiences because I watch your software in use and wonder why you didn’t.

Every company that plans, designs, builds and markets products associated with computers relies on tools and specific methodologies for testing performed in-house. From the day ideas are born and scribbled on white boards or napkins at the bar after a few beers, the journey begins. Investors are contacted, or at least fantasized about. Overtime is demanded by everybody. Test plans are written. User personas are developed. Salespeople write press releases. Business analysis, requirements gathering, content writing, and that hefty investment in digital marketing are all part of the routine. A great deal of time, money and after hours drinking goes into the final product.

What is not on the list is real life testing with people in their native habitat.

Please consider performing tasks in real life during development and before releasing to production.

Because when that brilliant product, redesign, enhancement, rebuilt for mobile, awesome offering to the world rolls out and doesn’t work, all hell breaks loose.  Managers point fingers. Programmers insist they were simply doing as they were told.  (They are not instructed to think.  Just code.) Reputation management experts are called in to repair the damage. Marketing and accounting personnel are banging their heads on the wall. It’s chaos. It’s a scenario happening on a regular basis.

 

If you are a programmer, the stories you could tell would be perfect for Dead Men Walking plots. Because you were asked to be zombies who code.  You were even paid good money to be Dead Dev Walking.

In 2013 when Healthcare.gov launched, it was a total usability flop. It was so dreadful that my before and after screenshots of the testing I performed after its launch provided comic relief in my talks at conferences on website usability. Clearly, nobody was properly managing the project. Its failure to work without torturing users was catastrophic and yet one company sounded the alarm and were ignored.

“On Tuesday The Washington Post reported that the Obama administration knew there would be problems with Healthcare.Gov website as far back as March when they were presented with an analysis by McKinsey & Co.”

The article  went on to say that “flags were definitely raised throughout the development of the Web site … But nobody anticipated the size and scope of the problems we experienced once the site launched.”

Wait. Nobody anticipated? Did nobody TEST it first?

How many millions of dollars were invested in a product that ignored user testing, usability guidelines and in the case of a .gov site, Section 508 compliance? It was handed off to another company and 3 years later the website remains confusing to use for the people who need the help the most. Who are they and why are they ignored from the design?

How is it that the practices and disciplines performed by software developers and website designers have remained unchanged, while computer devices and user habits do?

Extreme demands placed on software engineers to learn more programming languages, rewrite code, and deal with illogical deadlines contributes to the disasters we have all experienced online. How we use stuff is not factored in because project managers are not taught or encouraged to go beyond mental models and user personas. The general public is unaware that their real life, individual, unique experiences are not designed for. Instead, a user persona is written up and used to act out your role to help developers and designers map out the user interface and functionality or representative data is gathered into a use case for the sake of laying down code.

It’s improv.

User Personas On Stage

A dozen years ago I created user personas as part of my usability testing methodology. I developed a fun way to teach clients the value of testing and considering real people by creating “user characters” that acted in specific situations so that the client could see how their website or shopping cart was actually used out in the wild.  I decided on this method because companies weren’t investing in user testing and their heads were wrapped around search engine marketing rather than user experience.  Most had never heard of user personas or mental models.  I stopped offering this service when it was replaced with SEO tools, Google Analytics, heat maps and click tracking to study performance.

It is easier and cheaper to automate and guess what people will do.

Research Drops Clues But We Are Unique and Unpredictable

Men click there on an image for an automobile, while women click here. Therefore, ALL men click there and ALL women here. End of story.

Neuroscience and behavioral studies contribute data, providing more metrics to help guide decisions such as where to place a call to action button or what color to choose. However, how do you account for the person unable to see colors or who just forgot where they were and where you placed the call to action button?

There are sheer mountains of research on the user experience of things done on desktops and older operating systems.  The same is true for search engine marketing and the algorithm chasers who stalk search engines for changes.

I applied for a user interface position with a company I was willing to give up consulting to work for and when I asked them what they use for user experience requirements was told, “We follow Jakob Nielsen’s 10 guidelines.”  Those 10 guidelines were introduced in 1995.  I replied, “He has said he has over 2000 usability heuristics and that was a dozen or more years ago.”  I didn’t get the job.

There is no job description real life user experience testing, planning, requirements and heuristics. Companies are not looking for user testing in real life, with actual people performing tasks in various environments using the tools and applications they need to accomplish the stuff they want and need to do.

An article by the Ministry of Testing, called What Do Software Testers Do? contains a list of what testers do. Accessibility testing is not there. If you are a human who can’t use a mouse, has short term memory loss, does not see with 20–20 vision, or has a disease that creates hand tremors, who is testing for you?

Under the category called “They Do Testing Type Activities”, not a single activity exists called, “Take it outside and use it”, or “Use it with real people”.  And here’s my favorite activity from the article, “Putting myself in our user’s shoes to test products”.

Pretending to Be UX

How does an off-shore programmer know what it’s like for anyone to use software created for use outside their country? I shared an article on my Facebook page recently that described how people in India use mobile devices because American software developers and ecommerce companies don’t even think about things like that and yet they claim to reach a global market.

How do software testers, most of whom continue to be men, have any idea what it’s like to perform tasks that women do? We don’t even think the same, let alone use software the same way. I think about real world user experiences because I watch your software in use and wonder why you didn’t.

I think about crazy things like where the user’s gaze goes when they are attempting to purchase from an online store from a mobile device on a noisy packed train surrounded by a rowdy, “I’m not home in front of my parents” Girl Scout team taking selfies.

I think about your nifty cool tools that claim to work on mobile devices and how well they perform at a conference with spotty Internet connections and your customer has a client emergency.  They are likely stressed.  There’s the hunt for a connection, a quiet place to think, the rush to solve a problem because the party on the other end is freaking out and they had 2 glasses of wine.

This is never on a whiteboard.  The whiteboard version has complicated navigation, ads, slow loading images, confusing tasks, hidden calls to action and the mobile version is still in the Director’s office whiteboard sketched in black ink with smiley faces.

Does your company include software functional or web front end design requirements created for human conditions such as pain, addiction, short term memory loss, color blind, hand tremors, extreme stress, poor lighting and noise?Most clients provide me with their target user demographics and analytics, but never explain the situations for use.

How we use computers every day is categorized and based on assumptions or old data. User testing, whether by video or survey questions, is not performed in real world situations.  Testing environments are quiet and controlled. Nobody thinks about my realtor friend who conducts business sitting at a rustic wooden table on the outside porch of a Starbucks in a small town using just his cellphone distracted by people watching and the occasional young person who stops by to chat.  He sells houses and land.  Have you ever seen realtor software?

I sometimes meet with clients outside the office. Whoever designs things like changing DNS, installing WordPress and purchasing server space is missing out on the joys of using their services and products in the vast wilderness of the local Panera Bread with public free Wi-Fi.

Do you send your employees to conferences or seminars and expect them to work from the conference venue or hotel? Please know that the software your employees must use was not built to work in these environments. A real life user experience tester would be like a travel scout who goes out and tests every hotel, conference venue, plane, airport, taxi and train and bar to performance test.

What a value proposition to be able to claim your product or service passed!

You Are Paid To Build For Failure

How many times have you been to a medical facility where the doctor, nurses and receptionists are drumming their fingers, rolling their eyes, and tapping their feet waiting for their healthcare software to load? Or they say, “We just got this new software and I have no idea how to use it.”

Every time the user experience changes requires re-testing outside in the real world with the people who use your software or website. If you are a mobile application developer, you already know abandonment rates are rising. Every time new technology is introduced means it’s also an opportunity for watching people learn to use it. For example, the computer chips they are putting into credit cards creates an entirely new user experience and change in habits that create embarrassing experiences for customers.

Like when I used a department store card with the new computer chip. The store installed new card swipe machines, creating a new set of steps. First you stick the card into the slot and leave it there, but first you have to hit “O”. Which I did. Twice. I got an error each time. It turns out they really mean they want you to push the green enter button with the “O” on it, not the letter “O”. And then, after it accepts your payment, it beeps loudly as if to scold you for paying the store.

The Real Life User Tester would be a hero.