Whys, Dos, and Don’ts While Conducting Usability Testing in India

Again, I am picking this blog entry from Sneha's post on Kern's blog.

While conducting user research in India, work closely with your local partners. Here are some do's, don'ts, and whys that will help give you a perspective.

Do's	Don'ts	Whys
Discuss goals and the reason for each goal before your local partner starts recruiting.	Do not just send a document with your requirements.Reasons:Different cultures interpret English differently. Your requirements may have different cultural implications. Having this discussion will have you and your Local Usability Company (LUC) on the same page from the onset.	In one of our projects, client wanted to test their product on parents with babies. Client goal: To understand product usage of recent parents with stable income and who provide for a family. Our understanding before discussion: To test how parents interact with the product and their child simultaneously.
Explicitly explain each participant's profile.	As stated above, do not expect the document to do all the talking.	Client stated requirement: Parents with babies. Their intended requirement: Infants Our understanding before Discussion: Kids Kern was able to clarify the participant profile because of early discussions with our client.
Request your local partner to guide you with user profiles.	Do not assume the goals of user profiles to remain consistent throughout the world.	Client's Expected User Profile: Users from SEC*-C and D who use low-end mobile phone. Indian Reality:  SEC C and D have high aspirational value. Many have high end mobiles as for social status value. * Socio-Economic Class
Ask local partners about the language of the study according to user profiles.	Do not assume the language of usage. Let the LUC be the best judge of thisIn India, not every Socio-Economic Class (SEC) requires the usability testing to be conducted in the native language.	Client Expectation: Conduct study in native tongue. Indian Reality: Certain sections communicate better in English. Reason: India has 29 official languages. 1 national language. National language is not spoken fluently by most in south India. Hence English is the preferred language.
Ask about appropriate location for the study.	Do not force your understanding of the most ideal location for the study.	Client Requirement: Conduct the study only in the metropolitan cities.Kern realised that the profile could be matched in any Indian city. Indian Metro Reality: ·   Anywhere is far away in metros, commute is a huge problem. ·   This affects participation. ·   For participants, attending a UT during the work week would be sacrificing an entire work day because of the commute.
Ask LUC to localize the protocol. Listen to what LUC says about how users may behave or why some tasks may be relevant or not.	Do not, for the sake of consistency, be rigid about modifying the protocol.	In our experience, not localising the protocol results in Moderators spending double the time explaining the tasks and requirements to the participant. The questions were not relevant to the Indians context. This skews the results, giving false positives or negatives.
Be present during the study — some results may be startling! Ask LUC about the possible reasons for such results.	Do not expect the recordings to help you 'catch up' on the proceedings.	Being present also ensures immediacy; clarifying doubts or key observations immediately. This ensures that you get the core issues as and when the product is being tested.
Have a quick de-brief session with LUC after each study — make detailed notes. It will help you understand local culture well. Ask the LUC to provide cultural insights as a part of the deliverables.	Don't expect the details in the report to give you a complete understanding. De-brief will give you that extra bit of understanding that a formal report cannot.	Kern ensures that clients don't leave with just the numbers or even its implications but with an understanding of the whys behind the implications.

In addition, ask your local partners to suggest solutions to the problems found. You may find solutions that are contrary to your thinking. Ask reasons for such solutions.

Kern offers usability testing services in India

Kern has been providing usability testing services to its clients and other usability consulting companies around the world. Till now, we worked with clients who knew that we provide this service and somehow we never announced it on Kern's website. 

However, as we have perfected the service over many years and we have excellent clients, we today announce usability testing in India as a service on our website. Kern's usability labs are located in Hyderabad, but Kern can run the studies in Bangalore, Mumbai or any other city, town, or village in India.

http://kern-comm.com/usability_usabilitytesting.htm

We are also announcing that Kern's full digital usability lab is available for rent to any company. Please contact me for any requirement.

Kern forms Joint Venture with HumanCentric

Kern Communications Pvt. Ltd., India's leading user research and usability consulting company announces a joint venture with USA's HumanCentric, an award-winning and integrated product design company committed to make consumer-friendly products by studying and understanding the end-user.

This joint venture comes in at a critical juncture for both the companies. HumanCentric and Kern partnership will help both the companies extend their expertise and services to both the geographies. As a part of joint venture, Kern will help HumanCentric's worldwide clients with the necessary cultural knowledge and user research expertise to design mobile, automotive, medical, consumer, and software products for the booming India market. On the other hand, HumanCentric brings its product design and product engineering expertise to help companies in India take their products global.

Kern and HumanCentric share similar ideologies and complementary skills while user-centered design and user research are the core competencies of both companies. The joint venture brings in increased reach across geographies, global expertise, consolidated experience, and pooled resources that will help Kern and HumanCentric offer solutions to clients globally.

"The joint venture with HumanCentric brings together two companies with complementary skill sets, vision and commitment to excellence to provide world class services to clients. HumanCentric is well known for its team of highly skilled professionals and award winning design solutions. This joint venture will help leverage Kern's skills and expertise in usability and help HumanCentric gain better understanding of the Indian market." says Ripul Kumar, Director of Kern Communications.

"This joint venture with Kern Communications represents a wonderful opportunity for HumanCentric," says Barry H. Beith, CEO and Chief Technical Officer of HumanCentric. "Not only is Kern Communications a very talented and high quality design house, but their skill set, perspective, and attitudes are very complementary and compatible with our team. They provide the necessary cultural knowledge and perspective that gives HumanCentric the ability to provide the proper research and design support needed by our US clients to successfully introduce products into the emerging marketplace in India, a marketplace that is 350 million users strong. HumanCentric, on the other hand, provides Kern Communications with a bridge to potential clients in the United States and a strong partner to provide Indian firms with design support for American and European clients. Companies such as Tata Automotive, which just bought Jaguar and Land Rover from Ford, can now turn to Kern Communications in order to bridge Indian and U.S. product design needs. "

About Kern

Kern is a leading usability consulting and learning solutions company in India. Kern works with top financial institutions, start-ups, established software product companies, online gaming companies, and mobile phone makers to help them make their products easy and pleasurable to use.

Kern designs solutions that are easy to use and fun to learn. Kern's strengths lie in understanding users and translating to useful, usable, effective, and successful solutions. Kern consultants are experts in the field of usability, user research, and pedagogy. Today, Kern boasts of an impressive client list and diverse project experience. Kern helps companies understand their users' life, work, goals, and motivations.

Making things "simple, useful and usable" is our business!

Press Contacts

Geeta Bose
geeta at kern-comm dot com
+91 40 401 71313
http://www.kern-comm.com

Why does usability research requires only a few users?

Usability research is usually qualitative, or driven by insight (why users don't understand). Qualitative research follows different research rules to quantitative research and it is typical that sample size is low.

The end result of usability testing is not statistical validity per say but verification of insights and assumptions based on behavioral observation - quantitative vs. qualitative.

·       We are looking for behavioral based insight, like what people do.

·       Statistics tell half the story and often are devoid of context, like why did they fail?.

·       Research clearly shows that even with low numbers, you can gain valid data.

·       Usability testing is being used industry-wide and has been for past 25 years. Experts, authors and academics put their reputations and credentials behind the methodology.

As mentioned earlier, Usability research is behavior-driven: You observe what people do, not what they say.

In contrast, market research is largely opinion-driven: You ask people what they think and what they think they think. For this, big samples are needed for market research. This is why phone or web surveys require hundreds or thousands of responses.

Behavior-driven research is more predictable. Basically, if 10/15 users are confused you can assume that many more will also be confused as well.

How are Focus Groups different from Usability Testing?

This was originally written by Mini and I have again shamelessly copied this from Kern's Blog.

In today's fast paced life, people tend to avoid products that frustrate them. Satisfied customers would be ready to buy good products and services. This also helps in word-of-mouth marketing.

Used together, focus groups and usability testing help in building great products and services which are highly desirable, satisfying, fun and easy to use and also worth being recommended  to others. However, both, focus groups and usability testing have different focuses and must be used in conjunction with each other. Focus groups and usability testing must never replace each other.

A focus group speaks about the extent of promises that can be made to the customer while usability testing helps shaping the products based on those promises. Usability testing is special as it focuses on eliminating end-user frustration by turning it into satisfaction and pleasure.

This article is a continuation of a series of comparisons of various user research techniques. Here I've contrasted focus groups with usability testing.

Focus Groups	Usability Testing
What is the meaning of focus groups? Focus group is a brain storming session group administered  a moderator. The goal of a  focus group is usually to get a variety of  broad  opinions of what people want a product to do for them along  with what will drive the end users to buy the product?  This helps the moderator to begin considering the features of the product. Focus groups are  opinion based and their motivation is to find out what the desires and wants of the users are. .	What is the meaning of usability testing? Usability testing is a one-user-at-a-time testing by users but moderated by a seasoned usability professional. While the user interacts with the product in a structured manner, the usability professional notes down the stumbling blocks, hits, and reactions of the users while they use the product. This help usability professional to guide product development decisions. The key focus is what will make the end users of the product to learn about and use the product easily. Usability testing is observation based and tells  if or not something works.
What is the output of focus groups? The output of a focus group is a collection of various opinions, attitudes and preferences of the users.	What is the output of usability testing? The output of usability testing is observational and behavioural measures based on the ease with which the users handle the products.
When should focus groups be used? Focus groups are best used in the early stages of product or service development to provide wide-ranging information on opinions and attitudes of the target  user group. This is used primarily when important investment decisions are to be made. Focused on opinions: Focus groups have many voices at the same time, so it is impossible to observe the users in action and record the steps. The moderator can, at best, get opinions about their  intended usage With this lack of focus and precision, it only elicits vague opinions. Consider what to build: The purpose of a focus group is to decide what kind of product to consider building. Decide Costs: A company uses focus groups to decide the extent of investment of product development. This results in large near term costs in the hope of future profits.	When should usability testing be used? Usability testing is used throughout the development process to help in finely focused development decisions and it also guides in the realization of the big picture. During the early development stages, paper-prototype based Formative Usability Testing is used to test ideas, navigation, and task flow. Post development, Summative Usability Testing is used to benchmark navigation, task flow, measuring user performance, errors, and subjective satisfaction. Focused on tasks: In usability tests, the users are asked to complete many tasks one after another. The difficulty with which the users complete the task is carefully recorded. This clear focus helps them to get more specific and actionable results. Making products better: The purpose of a usability test is to improvise the product to make it work better. Increase profits: The usability test is used to improvise the products resulting directly in cost savings, greater sales, and increased profits.
How many people do I need for a focus group? Focus groups involve 6-10 people interviewed together in the same location at the same time.	How many people do I need for a usability testing? Ideally about 5-8 users are tested per sub-group individually with usability professional.
How much would  a focus group cost me? To conduct a focus group, expenses will be incurred on recruiting, incentivising, focus group facilities such as a place where a focus group can be conducted, a place where clients and others can view the proceeding, and audio-video equipment must be readily available. Also expenses have to be incurred to carefully prepare the focus group protocol and also to analyze the findings. Focus groups are  less expensive when compared to  usability testing.	How much would usability testing cost me? Preparation of testing protocol, hiring a usability lab with one way mirror, recruiting and incentivising people from a target user group, conducting tests one-by-one, and analysis is time consuming and expensive. Usability testing requires lot more time to plan and organize. Usability testing is one of the most expensive and most effective usability evaluation methods today.
How long does it take to conduct focus groups? A focus group involves a detailed protocol scripting and  participant recruiting. However, it is not time consuming when compared to usability testing. Ideally a focus group session should not exceed 1 ½ hours. For a couple of focus groups involving 14-20 people, Kern would ideally take 2 ½ weeks to recruit, conduct tests, and present analysis.	How long does it take to conduct usability tests? Usability testing method  involves detailed protocol scripting, participant recruiting, and conducting the test one by one, followed by data analysis and report generation, it is a time consuming process. For a usability testing involving 2 sub-groups (about 16 users), Kern will take about 3 ½ weeks for the entire process.
What are the advantages of focus groups? Focus group is one way to gather information about a target audience that you have very limited information about. They help us understand users' opinion, attitudes, preferences and initial reactions. Group discussions often spark new ideas.They are excellent if you are looking for multiple and a wide range of opinions, ideas and points of views in a short period. Focus groups are cheaper and faster to conduct.	What are the advantages of usability testing? The results of usability testing give a truer picture of the real problems people encounter as they're derived from observations of real users.. They help us to understand the different behavioural patterns of the users while using the products and services. Videos of user sessions provide a very powerful persuasion tool for reluctant stakeholders. It's hard to argue with recordings of people in tears of frustration. Understanding points of frustration usually trigger practical solutions.
What are the disadvantages of focus groups? The findings do not lead to statistically significant results. As there are more members, one or two members' opinions usually dominate  others and this does not help in arriving at  clear conclusions.  opinions.	What are the disadvantages of Usability testing? Usability testing is expensive and time consuming. The validity of test findings depends heavily on identifying the right target group, accuracy of usability testing protocol to recognize key user tasks, and accurate user recruiting.

References:

http://www.macadamian.com/index.php%3Foption%3Dcom_content%26task%3Dview%26id%3D54%26Itemid%3D0

http://www.usabilitynews.com/news/article4062.asp

http://jobfunctions.bnet.com/abstract.aspx%3Fdocid%3D242848

http://www.zanzara.com/pdf/UsabilityVsFocus.pdf

Iomega StorCenter 150d: Dead or Alive?

14 July, 10:30 AM: Neoteric Engineer Turns Up
Copying of files is over - no exceptions! Neoteric engineer wants to take the box to the service center and update firmware. He says that he will take 2 days to finish the work.

14 July, 5 PM: Neoteric Engineer Brings the Box Back
We connect it to the network -- seamless! Where is the data -- it's all there -- safe. Excellent!

So, the story is simple: Iomega StorCenter actually updates the firmware without loosing the data -- that's great! However, people at
Iomega don't even trust their product to do it right or they just want to wash off their hands about any data loss -- your guess?

BTW, Iomega India did not give a new box as promised -- but I suppose
if the old one is back on the network with all the data intact -- why should I bother. I just fear the next firmware update.

Mr. Annamalai (or Mr. Iomega) please come out of your cushy air-conditioned office and spend time with small businesses to understand their problems -- it will be an eye opening experience.

Kern is now buying a Thecus N5200B Pro for critical backups. Kern will now use the Iomega 150d for non-critical temporary storage.

Iomega India: Pathetic Products, Bad Service

11 AM - Neoteric Engineer arrives to baby sit the copying process on Iomega 1Tb 150d. He goes in and out of our office throughout the day to kill boredom of copying files from one box to another.

4:37 PM - "Can't find mapped device"

On further interrogation, it is found that Iomega 150d decided to change its static IP address from 192.168.1.20 to 192.168.1.2 -- probably 150d got bored and wanted some fun! Started file transfer again. Neoteric Service Engineer disappeared long back.

7 PM - Copying is still going on. About 137 GB left -- I think it will take the whole night. Have lost more than 27 hours copying the data to another device. I wonder if it is going to take the same amount of time to put the information back on it -- just for a forced and silly firmware update!

Lessons from the story: 

a. Never buy or recommend any Iomega products to friends and foes. Just for a silly firmware update (which you are forced to perform) you will loose all your life's hard work. 

b. People in Iomega India don't care unless their asses' frying on burning charcoal.

c. If you want to update any firmware, please understand that the device will loose the most vital information -- especially Iomega's designed devices. I will never buy an Iomega home toaster if they make it! I like to keep my toasts on setting 2 -- always!

Previous Iomega Posts:

Day 1: Iomega StorCenter 1Tb 150d woes

Day 2: Iomega StorCenter 1Tb 150d woes: 10 July 2008

Iomega StorCenter 1Tb 150d woes: 10 July 2008

The earlier part is here:

http://ripul.blogspot.com/2008/07/iomega-storcenter-1tb-150d-woes.html

8:37 AM - Mr. Iomega calls up

a.  "Why did you put my name on the blog?"

Oh! Someone is really scared about his name on the blog and not worried about the customer woes. You have a broken product Mr. Iomega, and your only responsibility is to solve the problem.

b. "Would you take my name off the blog if I solve the problem?"

Iomega must move from IT products to street politics -- kudos for cutting a deal with the customer for providing service on a faulty product.

c. "I didn't call you an Idiot, why did you say so on your blog?"

You made me feel like a big one -- isn't it the same? Anyway, this clearly shows how much Iomega India is committed towards solving customers' problems. Make a customer feels like a big idiot is absolutely rude and unpardonable. Is anyone at Iomega India listening? Is Mr. Bhaskaran listening?

d. "Why did you go public? its not even a day since we know about the problem"

Based on Iomega India's history with us (2 months non-availability of product and without a backup), and the previous peace-talks with Mr. Iomega, it was foolish for us to wait for any service.

9:01 AM - Mr. Iomega calls up once again

a. The neoteric engineer will come to your office, will bring all firmware updates, and a 500 GB backup, and will back up your data, and update the firmware to complete the service.

Ah! That sounds good! Why didn't something like this come up earlier? Why was Iomega India so hostile earlier? Okay, Kern will give the service engineer a place to sit with an expensive revolving chair, a laptop computer (chances are that he will fail to bring one!), network access, tea/coffee, food during lunch time. And, anyway Iomega will levy service charges even during warranty period -- which we would have to pay up, isn't it?

The earlier response by Mr. Srinivas, service engineer of Neoteric was "Get the box here, whatever the problem is -- the box has to be replaced, it can't be repaired. We will charge you for replacement, and your data will get destroyed!"

10:02 AM - Neoteric Service Engineer calls up

Saar, there is no light (electricity/power) and I am trying to issue a 500 GB box, I will come surely saar.

1:45 PM - Neoteric Service Engineer calls up

I am finishing my lunch, I will come after 2:30 PM

2:40 PM - Neoteric Service Person Arrives

As expected the service person arrives with just a Lacie 500 GB new hard drive, without any diagnostic tools, and no computer -- how does he plan to go ahead with the service? Kern spared an old laptop. 

3:39 PM - Mr. Iomega calls up

Iomega India will replace the 150d box within 2 days -- but will not take care of data.

4:45 PM - Neoteric Service Engineer could finally connect to the drive and start copying our data to Lacie 500.

6:45 PM - The copying stopped, Iomega threw up an error "Unrecognized file name or too long a file name"

Excellent -- when it saved it had no problem, now it has! This game is addictive.

7:00 PM - Neoteric engineer leaves after we gave in writing that we will take care of "his" Lacie 500. 

7:45 PM - Kern starts data backup again from the beginning.

11 July: 10:00 AM - The data had stopped copying in the night. No errors! Restarted the whole process -- now by copying individual folders one by one.

All this just to update a silly firmware? Wake up Iomega and design good products. You have lost a small business that has growing needs.

VFS Fleecing Unsuspecting UK Visa Applicants

Here is the letter I sent to the honorable British High Commission after I came back from VFS center in Hyderabad -- a horrible experience! VFS people clearly told me that we are advising no visa to me as my photo does not confirm to specifications. Interestingly, the VFS notoriety is exposed -- British High Commission did issue me a visa (and my chin in the visa was clearly visible!) and that too in record time of 3 days -- wow, that's for great customer service!

----

30 June, 2008

To,

Sir Richard Stagg

British High Commission

Shantipath

Chanakyapuri

New Delhi - 110021

Sub: VFS fleecing unsuspecting applicants and giving out applicants' information to third parties

Respected Sir,

We appreciate your interest in promoting better business ties between UK and India. I am the director of user research at Kern Communications Pvt. Ltd., a 100% export oriented unit registered with STPI in Hyderabad.

I and a colleague of mine were at the VFS center in Hyderabad today (30 June, 2008) morning to submit our applications for a Business Visitors visa for UK. We both were surprised at the way VFS employees were fleecing unsuspecting applicants.

a. We both had a photographs that match ALL the required criteria perfectly - new photo, exact size verified by a scale, color photo, white background background, full face with no covering, etc. However, we were asked to get new photographs shot at the VFS center for Rs. 100/- a shot. 

When we asked them about the reasons, I was told that they cannot scan the photo as "your chin got completely cut in the scan". On the other hand, my colleague's photo was rejected due to "photo on non-glossy paper," however, the photo that was taken had no difference in the gloss with the photo that was produced. Both the above reasons sound baseless and seemed only to fleece applicants.

b. We were forced to pay Rs. 170/- for courier as extra. This information only available only in "Additional Services" in VFS website and is not mentioned in any routine document or webpage. When we enquired about this additional charge, we were told that VFS can deliver the passport at the counter only two days after the courier does, however if we want it speeded up by 2 days we must pay the courier fee.

I would like to ask a couple of questions to the honorable high commissioner:

a. I understand that the British High Commission delivers the passports at one time, but why should VFS deliver the passports two days after the courier delivery?

b. If the above is true, why does VFS or British High Commission add that visa facilitation charges to visa charges so that it does not leave a sour taste? or clearly specify upfront that there will be courier charges?

c. VFS is giving private and confidential information about the applicants to third parties that are neither UK government departments nor public authorities. I got an email from RKG Consulting, a private company, with subject "Sending Staff to UK: New visa rules" immediately after my visit to VFS center. I understand that I have not given my email address to any third parties except VFS who know about my visit to UK. This is absolute violation of my privacy and would like to register my complaint against this. I am attaching the said mail and its attachments as PDFs for your reference.

Sir, today's experience at the facilitation center and the breach of my privacy left a sour taste. I would request your esteemed office to look into the matter to ensure that VFS does not fleece the applicants and correct information is given to the applicants at all times. Please also advise if VFS has any right to pass applicants' private and confidential information to third parties?

Thanks and regards,

- Ripul Kumar

--
Ripul Kumar
Director, Usability Consulting & User Research
Kern Communications Pvt. Ltd.
http://www.kern-comm.com

* Usability in India *

Iomega StorCenter 1Tb 150d woes

Piggy on the railway track
Picking up stones,
Along came an engine
And broke piggy's bones.
"Oh," said Piggy,
"That's not fair!"
"Ha!" said the engine driver,
"I don't care!"

A tiny little user research company's bones are broken - again! The Iomega StoreCenter 1Tb storage on the network died one more time, breaking Kern's bones once again. And when Kern cried "That's not fair;" "Ha! Ha" said Iomega, "I don't care!" Oh, this is not the first time Iomega India said "I don't care!" They have said it earlier too!

Kern bought a wireless 1Tb StorCenter about a year and half back from the neighborhood Iomega retailer. Kern was happy! Now we could store all the projects backups as well as the prized photos and videos too. The happiness was shortlived! The NAS stopped serving files one fine day -- just 6 months after it was purchased.

The local support guys Neoteric were called. They promptly came and picked up the device on a bicycle and dropped the device on the way to the support center -- the already broken device was now beyond repair!

Many mails to the CEO Paras Shah, screams to the engineer Srinivas, persuasions, cajoling, heart burns, and 2 full months later, Neoteric agreed to replace the device with a new one. But, here comes the catch, they tried to upsell me the newer version 150d -- we refused. Then magically one day they dumped the new 150d in our office (with no box, no documentation, no nothing -- just the box).

All our data was back, thought the little piggy! But the data was not to be seen. The piece was fresh and without our data. Kern had lost ALL its data. Many years of data had to be found again.

Kern slowly picked up the stones and moved ahead. The new box found its place on the network and people started using it again. Another 11 months passed, Ashim cried one day "can't find the network storage!" And, since then no one could find it.

Along came an engine
And broke piggy's bones.

I went to www.iomega.com and tried to find out the local support numbers. The website is designed for people not to find support numbers, but my persistence paid off. I found the number after full 10 minutes of trial and error (and BTW we usability test websites... we usually find things faster than others!)-- a 800 number -- a toll free number, wow! Hey, the toll free number is a farce -- only people with BSNL phones can dial the number. There was no phone number that anyone can dial -- kudos to Iomega India. Thank you Iomega for being unreachable -- don't expect people to subscribe to BSNL service or run to a BSNL phone booth just to contact you.

"Oh," said Piggy,
"That's not fair!"

Neoteric engineer was called up and he shrugged off his responsibility by giving out Iomega's number -- some Mr. Annamalai in Chennai. His phone number seemed a VIP number, I assume he is a "high ranking official". I promptly called the gentleman.

Here are some excellent responses from Iomega:

a. "We sell thousands of boxes of NAS, why do we only hear Kern's complaints?" Mr. Annamalai, you hear Kern's complaints because they are the only one who could somehow find you. They are the only micro business who had the courage to buy NAS. They are just one of the companies without an IT guy who will magically "get around" things without contacting you.

b. "Kern does not want to call the call center not do they want to use online chat" Iomega must supply a BSNL phone line to all its customers to call them free with each product. Iomega US-centric people please understand that India is not US -- no one needs a toll free number! People need a number that ALL operator people can call. And, how on earth can one get to the online chat? Only PhDs can do it, I suppose.

c. "Your drive died because you idiots (didn't say but made me feel like a big idiot!) did not update the firmware" Sorry Sir! Like all small businesses around the world, we have a battalion of IT people to take care of firmware updates of each and every device on earth. Sir, we are sorry that we bought an IT product without hiring an IT guy first. Now Iomega will drive the way we do business. Next time we buy an EPABX, we will first hire a telecom engineer -- my promise.

d. "The firmware update will erase all your data, that's the way!" Yes, Iomega behave exactly the way it was designed. But some companies design it slightly differently: when the Cisco-Linksys router that runs our network silently updates the firmware, we do not loose network settings. When our Indian EPABX updates the firmware, the phonebook is not lost. Yes, but I think Iomega is highly advanced -- it starts from a clean slate each time. Mr. Iomega, please understand, micro businesses like ours use the drive as backup as we can only afford this -- and if you expect to loose all data with each firmware update -- you can go and jump! We will find a better solution, thank you.

e. "This product has a three year warranty" Oh that's a sexy thing to say! If you cant get my data back, any warranty is a hog wash. Anyway, you are not honoring your three year warranty for us. You are still honoring the one-year of the previous one.

f. "I am calling to tell you that I can't do anything" Ya, my pop will do it, if you can't! He will come with his plumbing tools -- probably that will work.

g. "Why don't you back up all your data on another drive and leave it through the night?"
Good idea Mr. Iomega. Let me run to the neighbourhood chai shop, buy another Iomega 1Tb box, back up all my data on it by sitting through the night (actually longer for my more than 500 GB of data) and then do the upgrade. And, do this everytime Iomega thinks that the firmware needs to be upgraded. If I don't do it, you know what, the box will die down as it did.

h. "We have given USB ports, just connect your computer and the problem is solved"
If my memory serves me right, I bought a NAS -- which to my limited brain power means that the box is designed to be on the network -- this one refuses to be on the network. Do you think my peanut-sized brain is missing something?

"Ha!" said the engine driver,
"I don't care!"

Usability Testing Versus Expert Usability Review

This article appeared first at Learnability Matters, a blog by Kern Communications. It was orignially written by Sneha Nair, and edited by yours truly. And, I have taken permission to shamelessly copy :)

Sometimes we want to do the right thing to increase our application’s usability but don’t know how and where to begin, what method is appropriate to maximize an optimal outcome.

We’ve heard of many methods from usability testing, focus group, expert review to contextual enquiry but what’s the difference and when do we use it? Contrasted below are two usability evaluation methods that are often used.

Usability Testing	Expert Usability Review
Who conducts them? Usability testing involves getting people from the target audience to evaluate your application while performing specific tasks. Usability experts watch the participants performing the tasks and analyse. Kern provides usability testing services in India. Kern helps companies around the world understand Indian users and their usage patterns.	An expert usability review is performed by usability experts. The usability experts inspect an application to identify potential usability problems. The experts review the applications based on published research, experience in watching user behavior in the usability lab and design heuristics. Kern follows a unique methodology for expert evaluations that gives a rigour and actionable results to clients. This methodology is a mixture of Heuristic Evaluation, GOMS, and Cognitive Walkthrough.
How many users do I need? Usually 5-15 respondents are enough to find key usability issues in a usability test. Kern helps clients understand how many users may be optimally needed for a test. Kern will also help clients understand where it would be ideal to conduct such a test.	Expert usability evaluation requires absolutely no users. Usability experts perform the evaluations. Using the traditional expert evaluations methodology, it is important to have more than one expert to perform the evaluation. Number of problems found will increase when the number of evaluator increase (Kantner & Rosenbaum, 1997). The ideal number of evaluators is 2-3. Using Kern’s unique evaluation methodology, one expert evaluator can perform the evaluation without missing problems.
How much would it cost me? Usability testing requires more time to plan and organize. Preparation of testing protocol, hiring a usability lab, recruiting and incentivising people from a target user group and conducting tests and analysis is time consuming and expensive. Usability testing is the most expensive usability evaluation method.	An expert usability review is much cheaper when compared to usability testing. It’ll only take a few days to evaluate an application and write a report. You neither need to hire expensive labs nor recruit users; costs are significantly lower. With Kern’s unique methodology, we reduce the time taken for an expert usability evaluation, therefore saves you significant costs.
How long does it take? Usability testing method involves detailed protocol scripting, participant recruiting, and conducting the test one by one, followed by data analysis and report generation, it is a time consuming process. For a 15-20 people usability test, it usually takes 4-6 weeks, however Kern requires 3 weeks time – start to finish.	Skilled evaluators can produce high-quality results in a limited time. At Kern, it usually takes about one to three weeks for an expert usability evaluation including a report of findings and recommendations. The time depends upon the complexity of the application/website.
What does it help me find? Usability testing is better at finding issues related to special domain knowledge of users and their task flows – what path do they follow while performing a task. If the usability testing protocol is designed well and covers key user tasks, user testing helps you find major usability problems.	Expert usability reviews tend to find high level breaches of design rules and consistency. Analysis of interface architecture and page flow efficiency including navigation bottlenecks. Heuristic evaluation helps you fine minor usability problems. Though heuristic evaluation helps identify some major problems, the number of minor problem still outweighs the major problem. With Kern’s unique methodology, evaluators find all key and many strategic and tactical usability issues. This helps clients’ schedule implementation.
What are the disadvantages of usability testing and expert evaluation? Usability testing is expensive and time consuming. The validity of test findings depends heavily on identifying the right target group, accuracy of usability testing protocol to recognize key user tasks, and accurate user recruiting. Kern guarantees its clients accuracy of usability testing protocol and identification & recruitment of right users.	False positives: Research (Kantner & Rosenbaum, 1997) shows that 43% of the “problems” that were identified by the heuristic evaluators did not turn out to be problems at all. A lot depends on the reviewer’s level of experience. No two usability experts will find exactly the same issues so the method is also open to practitioner variation. The biggest problem however is simply that they sometimes miss the real problems that cause users to fail tasks. This can be especially true when the target audience has a particular skill set. With Kern’s unique methodology, false positives are significantly reduced. Unlike other firms, Kern always uses experienced evaluators to perform evolutions.
What are the advantages of usability testing and expert usability review? The results give a truer picture of the real problems people encounter because they’re derived from real users in the first place. Another advantage of usability testing is that there’s less conjecture and feedback comes straight from the horse’s mouth. Videos of user sessions in particular, can provide a powerful persuasion tool for reluctant stakeholders. It’s hard to argue with recordings of people in tears of frustration.	Quick and cost effective. Evaluators are usually specialists and their results often match user testing. Evaluators find the almost the same issues found in user testing. Sometimes they spot minute usability problems, often missed by user testing.
When do I use usability testing and when do I use an expert usability review?Usability testing is recommended for use later in the development process, when the extra value of finding the problems real users may encounter justifies its cost and time. Kern recommends summative usability testing only: When the application is used by domain experts When the user groups and tasks are clear During later part of development after formative tests and expert evaluations are done The users (and evaluators) are from a completely different cultural group from the designers The management must be convinced by using a proof	People often use expert usability reviews early on to straighten up their design in preparation for usability testing. Kern not only helps evaluate the application but also helps them give design directions.

References:

http://www.useit.com/papers/heuristic/ http://www.usabilitynews.com/news/article3616.asp http://www.userfocus.co.uk/articles/expertreviews.html http://www.oclc.org/policies/usability/testing/default.htm http://kmvirtual.com/2008/05/26/heuristic-evaluation-and-user-testing/ http://www.teced.com/PDFs/sigdoc97.pdf

3C's to take care when blogging about work

"Publish or perish" is repackaged in a new wine bottle by many corporates by encouraging employees and associates to blog about work and life. Many people see this as a way to publish and distribute best practices, publish amateur papers which otherwise may not get published in established journals, or talk about experiences.

Many people see blogging a way to become famous overnight while others try to make it a living resume with all gory details about projects, clients, etc. While we all do have a right to publish, we need to be careful while we do so about work related stuff. Remember, all the work that you do in a company, the intellectual property either resides with your company or its clients.

You must be careful about the the 3 Cs before your publish: Company, Competitors, and Clients. I am putting together some questions within each category that people must ask before they write an blog entry or just before publishing.

Company

a. Does the article contain description about processes, methods, or workflows proprietary to my company?

b. Does the article contain information that can give away my company's future directions, future products, or services?

c. Does the article contain references to the financial information about your company?

d. Does the article contain any information that will embarrass the company in any way?

e. Does the article contain any information that you remotely think is confidential for any reason?

f. Are you criticizing your company or your colleagues in any way?

Competitors

a. Does the article contain information that your company's competitors may benefit from?

b. Does the article refers to named competitors or identifies competitors in an indirect way (like "large mobile handset company in Finland")?

c. Does the article talk about yours or your company's capabilities that may benefit the competitors?

Clients

a. Does the article contain specific information that identifies specific clients or projects?

b. Does the article contain references to any project done at any point of time with any client? These can be methods, processes, workflow, blueprints, images, etc.

c. Does the article contain any information that is proprietary to your client and not a public knowledge?

d. Does the article contain any information that reduces or eliminates your client's selling or competitive ability?

If the answer to any of the questions is Yes, please do not hesitate to either change the article or run it in writing by a director level person in your organization.

If you do want to become famous, do blog wisely -- success is seldom overnight!

Market Research Vs. User Research

Market research and user research are analogous to each other but have completely different goals. They sometimes share common goals and inform one another.

Both disciplines have different approaches, their own strengths, and weaknesses. Which method to use and when to use it depends on what problem you are trying to solve.

Marketing research may start this effort with demographic data, while user researchers find information that challenges and qualifies the segmentation. Both types of research have a role in innovation; both can find gaps that may drive new product ideas.

Here is quick comparison of each method:

Market Research	User Research
Goal - Buying Patterns: Aims to understand buying patterns: understands who will buy which product/services and peoples opinions about a product/service.	Goal - Usage Patterns: user research focuses on usage patterns: how and in what circumstances users use products and services, and observes what people actually do.
Markets: What products or services people are willing to buy?	Usage: What are the usage patterns of a specific product or service? In which environment(s) do they use?
Customers: Which segments of people are willing to buy a defined product or service?	Users: Will specified use a specific product or service? What are the key roadblocks for using a product or service?
Competitors: Who are the competitors? What are the key success factors in the given industry?	Comparison: In which way people use competitors products differently for doing same task?
Attributes: What features and attributes of a product or service is important for a buying decision?	Implementation: How is a feature or group of features implemented? Efficiency, Learnability, Memorability, Errors, Subjective Satisfaction.
Marketing Department: Conducted by marketing department.	User Research Department: Conducted by usability or user research department.
Opinion Based: Aims to understand what people's opinions about products and services are.	Observation Based: Observes what and how people interact with products and services not what they say.
Big Sample Size: Needs huge number of respondents to get statistically accurate results. 800 ~ 2000	Very Small Sample Size: Need only 5-15 participants in a particular group for statistically relevant as well as for uncovering most usability problems.
Geography Based: Buying patterns change across smaller geographies. So research is needed in many cities, town, and villages.	Culture Based: Usage patterns change with change in sub-cultures. So, research is not needed across similar cities. The results are going to be similar.

How Does Each Type of Research Help the Other?

a. Market research finds out which users may buy a service or product. User research focuses on product/service usage by specified users.

b. Market research narrows on which products/services people may buy. User research focuses on how easy is to use the important ones.

c. User research uncovers unusual usage patterns of users. Market research can then conduct studies to find out if that is rampant and if products need to be designed catering to certain usage patterns.

References:

a. Market Research and Usability

b. What is a usability professional's role in conducting traditional market research?

c. Marketing and User Research