Usability research is usually qualitative, or driven by insight (why users don't understand). Qualitative research follows different research rules to quantitative research and it is typical that sample size is low.
The end result of usability testing is not statistical validity per say but verification of insights and assumptions based on behavioral observation - quantitative vs. qualitative.
· We are looking for behavioral based insight, like what people do.
· Statistics tell half the story and often are devoid of context, like why did they fail?.
· Research clearly shows that even with low numbers, you can gain valid data.
· Usability testing is being used industry-wide and has been for past 25 years. Experts, authors and academics put their reputations and credentials behind the methodology.
As mentioned earlier, Usability research is behavior-driven: You observe what people do, not what they say.
In contrast, market research is largely opinion-driven: You ask people what they think and what they think they think. For this, big samples are needed for market research. This is why phone or web surveys require hundreds or thousands of responses.
Behavior-driven research is more predictable. Basically, if 10/15 users are confused you can assume that many more will also be confused as well.