Printable version of lesson; return to section index when finished.

Evaluation

Introduction

This lesson covers site evaluation. If you are attempting to create good websites, it's obviously vital to know which sites are good and which aren't. Evaluating your site lets you figure out where it might be lacking. One way to carry out evaluation is with a "checklist".

Evaluation

Evaluating websites means determining whether or not they are "good". Normally this fits in with what we have discussed in this course (e.g., a good website must have worthwhile content) but sometimes there are additional goals depending on the specific website. For example, one goal of an online CD store would be to sell CDs: this means particular attention might need to be paid to the ordering part of the site.

Why evaluate?

Evaluating your own site can help you spot problems and make appropriate changes.

It helps to have somebody else evaluate the site for you (see last lesson for discussion of why this is beneficial), but you can probably gain some benefit even if you go through the procedure yourself.

Procedure

One way to evaluate a site is by using a kind of checklist. These checklists can cover various different aspects of design - for example, you could use a checklist of common mistakes, simply to ensure that you didn't make any of them.

Usability testing, discussed last time, was another form of evaluation for the specific (or general, depending on your point of view) issue of usability. You can use checklists to evaluate usability as well, but they aren't a replacement for usability testing.

Types of checklist

You can use checklists to evaluate websites in different ways, two of which are:

Both are useful.

Comprehensive evaluation

This approach does cover the entire area, but requires some work to organise. Coming up with a comprehensive checklist can be difficult.

Later, I will try to present an example checklist based on what we've talked about in this class.

Spotting problems

This second approach is easier: there are many "top 20 Web design mistakes" lists available, any of which can be used as a checklist to make sure your own site doesn't fall into those traps.

In cost-benefit terms, this can be more productive because it is very easy to do and it should help you find any obvious or serious blunders. So if you are short of time, this approach may be the best one.

Later, I'll give an example of this too.

Comprehensive evalution

You can draw up a checklist for comprehensive evaluation of websites, and then apply it to your site. It's possible to use the checklist in different ways, or to create a customised checklist depending on the needs of your site.

Creating a checklist

Creating a comprehensive checklist can be difficult. Depending on how much detail is required, the checklist could be anywhere from one page long up to, perhaps, a whole book worth.

I created an example checklist based on the principles covered in this class. Most of the questions in the checklist were drawn directly from the summaries of each lesson. So I spent only a few hours creating this checklist: a more exhaustive effort might produce better results. However it should be useful as an example.

(It may be useful to view the example checklist at the same time as reading this section: this link to the checklist will open in a new browser window.)

Customising the checklist

If your site has particular requirements, you may need to customise the checklist. For example, an ecommerce site might need to include more specific questions about ease of purchasing, providing users with information about other relevant items to encourage additional purchases, etc.

Using the checklist

You could fill in the checklist in two different ways:

Getting useful results

To avoid any bias, and make sure nothing is missed, it's preferable to get others to fill in the checklist as well as doing it yourself.

If several people fill in the checklist, you can then look at average or typical results, which are likely to be more useful.

Taking action

Once you have completed the checklist, you can look at the results to see in which areas your site still lacks, and then fix the problems (and go through the checklist again afterwards if necessary).

Drawbacks to this approach

There are a few problems with this approach:

However I feel that a comprehensive checklist can still be useful in helping you make sure your site does everything that it should.

Spotting problems

Another checklist-based evaluation approach can be easier to use but still provide much of the benefit. This approach is negative - with a checklist which lists things your site shouldn't be doing, not things your site should - and can be particularly useful because so many sites make basic mistakes. Lists of basic mistakes are available on the Web.

If you don't have much time to spend evaluating a site, this is probably the most beneficial approach.

Obtaining a checklist

Various sites on the Web include lists of "top ten Web mistakes", both from a technical and usability perspective.

I feel that this list from the infamous Jakob Nielsen is a fairly useful one. (Link will open in new browser window so you can view the list while reading.)

Using the checklist

The list can be used in the same way as before. Again, it's better if you get other people to evaluate your site.

Drawbacks to this approach

There are a few problems with this approach:

Overall, though, this kind of list can be very useful and it's relatively easy to go through. Many of the problems are practical and solutions can be directly implemented.

Case studies

These case studies were chosen almost at random, and can be evaluated based on either checklist.

Amazon.com

http://www.amazon.com/

internet bookshop

http://www.bookshop.co.uk/

useit.com

http://www.useit.com/

AskTog

http://www.asktog.com/

mozillaZine

http://www.mozillaZine.org/

Hacker News Network

http://www.hackernews.com/

BBC News

http://www.bbc.co.uk/news/

CNN.com

http://www.cnn.com/

Summary

Evaluating your site involves trying to work out whether or not it is a good site, so that you can remedy any problems.

There are different ways to evaluate your site - usability testing, discussed last time, is a form of evaluation. In this lesson we concentrated on one technique, the use of checklists.

Comprehensive evaluation

A comprehensive evaluation checklist tries to ensure that your site is good in each of various areas of site design. It is useful but can be difficult in two ways:

Spotting problems

A checklist for spotting problems lists common Web mistakes. You can use the checklist to make sure that your site doesn't make any of those mistakes. This is very useful and is normally quick and easy (since such lists are available on the Web) but:

Combining the two

If you have time you could use a combined checklist, with both types of question (aimed at spotting common mistakes, and aimed at highlighting things you should have achieved) which avoids some of the drawbacks.

Completing checklists

It is best if other people (preferably several other people) complete the checklists about your site, because they will not have the in-depth knowledge and bias about the site that you have.

References

Usability checklist for site developers

http://www.WebReview.com/1999/10_15/strategists/10_15_99_4.shtml

webreview.com's 10-point usability checklist.

Heuristic evaluation

http://www.useit.com/papers/heuristic/

Jakob Nielsen's paper on heuristic evaluation (specifically for ensuring usability), which is basically the same checklist technique. Includes a ten-item checklist.

AskTog: First principles

http://www.asktog.com/basics/firstPrinciples.html

AskTog (a usability column) gives a list of principles for the design of a website or Web application, which could also be used in a similar manner.

Top ten new mistakes in Web design

http://www.useit.com/alertbox/990530.html

More mistakes listed by Jakob Nielsen.

Top fifteen mistakes of first-time Web design

http://www.doghause.com/top15.html

A list by somebody I've never heard of, but it's quite reasonable.

Lynx viewer

http://www.delorie.com/web/lynxview.html

View websites as they would appear in Lynx - use this to check your site is okay for text-only and disabled users.