The invitation to ballot was sent to about 150 people. Ten people
participated, sending 51 mail messages. People either
sent one message with all their comments or one message for each of the
chapters they chose to comment on. There were
a total of 1905 comments/objections on the document.
The average number of comments was 191 with a range of 21 to 770 per person.
Of the 1905 comments the final version of the parsing system was able to identify
1900.
The comments were relatively evenly distributed over the
chapters, i.e. long chapters had many comments, short chapters only a
few. All but three chapters contained objections, the comments always
outnumbered the objections in a chapter by a factor of at least two.
By type, there were 1440 comments, 404 objections, 56 undefined (i.e. the type could not be automatically determined). Comments and objections were identified by the appearance of either word in the first few words of the comment in upper or lower case.
Placement of the comments was verified by checking the location selected against the final form version of the document. In general, section placement was satisfactory for all comments made that referred to a section. Page and line number comments were generalized to the section in which they occurred. The inability to assure non-intrusive placement of the page-line comments to the precise line within the copymarked document by doing string matches on the formatted text and the source code resulted in less than optimal placement of these comments.. It is possible that a better matching and placement algorithm might allow more accurate placement of page-line comments. Perhaps the use of multiple markers for signalling the occurrence of a comment would allow for more fault-tolerant comment placement. It should be noted that interactive use of CASCADE allows effortless placement of comments at a precise character offset. More importantly, because the placement is not calculated, but responsive to the placement by the commenter, it would be guaranteed to be accurate.