The verbal protocols that were gathered on this project covered four of the twenty hours that the editor used CASCADE. It should be noted that these observations took place during the first sessions so that many of the comments that were made were of the ``how do I ...?'' and ``what's this for'' sort. The information contained in the verbal record can be classed into two types: user observations and user recommendations. Broadly speaking the user expressed satisfaction with the system. Learning to use the system was observed and reported to be easy.
The suggestions for changes were varied and included some items that were
task-related and others that were more general. The editor felt that the
comments that had been submitted by the outside reviewers should have been
assigned sequence numbers by the mail parser so users could refer
to them easily.
This type of coding had historically been
used by the editors. Some other suggestions targeted the navigational
tools. The
mural display was singled out as being a very useful locator device when
attempting to navigate through a document. Given that the editors must address
each and every objection raised by the membership in a ballot situation,
it was
important that the location of each comment be highly visible. The color
coding
of the mural made the task easier than it would have otherwise been.
In addition to the information provided by the user, the observers made notes about the process during these sessions. In particular, it was noted that the user made frequent use of communications methods as he worked. At times he sent e-mail requesting clarification of terms or asking for directions as issues arose. It became clear that a useful extension for the system would be to further incorporate e-mail into CASCADE. While we will need to study the linkages further, at least two come to mind: 1. While reviewing a comment, mail should assume that invocation implies a note to a commenter about a comment, augmenting and speeding the process of making queries. 2. Mail parsing should include a subsystem to alert editors to responses to their queries and should `hyperjump' the editor back to the appropriate context.