Heuristic reviews can be a
valuable design and implementation
tool. But don’t mistake them for a full practices checklist audit, or a
usability test.
What Is a Heuristic Review
Jakob Nielsen and Rolph
Molich introduced the heuristic review in the
1980s as a faster and less expensive way to approximate the results of
a usability test. Unlike usability testing, heuristic reviews contain
pre-specified parameters that are given numeric scores by the experts
while role-playing a specific visitor scenario. It generally is
recommended that at least three experts conduct the review
independently and average their scores.
It is important to remember that heuristic reviews were developed in
pre-web days for use with computer applications. They provided a
framework to help usability experts identify and report common
usability barriers to developers and designers. When web sites came
into existence in 1993, heuristic reviews were applied to them almost
immediately.
By the late 1990s, the heuristic review became widely used and modified
as a method for comparing web sites to each other. Unlike usability
testing, the pre-specified parameters and numeric scores allowed a
single numeric value to be calculated for a web site. This meant sites
could be compared using the single score based on common parameters. A
number of consulting and advisory firms offered modified heuristic
reviews to companies primarily as a form of marketing bragging rights
for their web sites, even though there was never evidence that the
scores had any significant effect on visitors choosing a site.
However, even when the objective is to get the highest score, the
heuristic approach has serious drawbacks. First, assigning the numeric
values to the parameters is highly subjective. Second,
heuristic sets
have been adapted for standardized comparison not actionable response.
Recommendations for how to improve a web site for a higher score
require expert advice, rather than being presented as precisely defined
and cleanly organized practices in the heuristic review framework
itself.
Rolf Molich, said in a recent interview: “Heuristic inspections are
cheap, simple to explain, and deceptively simple to execute. However, I
don’t use this method very often and I don’t recommend it to my
clients… heuristic inspection is based solely on opinions.”(
1)
How to Use a Heuristic Review
This does not mean that
heuristic reviews have no value in the web site
design process. The heuristic review concept and framework do provide
value, in the context of their initial purpose, as a tool to help
designers and developers stay on track when making decisions on design
and implementation options.
Key visitor scenarios used with a heuristic review at regular intervals
in the design and development cycle act much like a carpenter’s plumb
line. They help keep specific design and implementation decisions
square with the objectives of the web site. However, heuristic reviews
should not be confused with a full practices checklist audit(
2 &
3) that
specifies all practice attributes known to be important or for a
usability test for verifying specific hypotheses.
There are a number of heuristic review sets available. The choice of
one set over another is less important than the development of the
scenarios used in the review. The scenario provides the description of
the visitor that helps the designers and implementers stay on track. If
the scenario is not well constructed and aligned with the business and
visitor objectives for the site, the result will be out of alignment
regardless of the review set chosen.
On the other hand, most heuristic review sets will produce similar
results when used by the same experts. Just having a structured
framework when applying the scenario is more important than which
structured framework is chosen.
Summary 
