Paper Prototype Results and Discussion
Paper prototype interviews with users led the team to discover difficulties and concept model discrepancies not previously considered. They demonstrated a significant number of problems with data formatting as well as a few confusing labels and control layouts. In addition, they showed that while current mail merge systems are document based, the concept model of a project based mail merge system would be useful to power users. They also proved that the designed interactions involving fields, conditionals, and previewing were intuitive and useful to users. The interviews were unable to show how experienced users would take advantage of the design's optimizations for re-using components of past merges and how intuitive the actual appearance of items on the screen would be.
Problems with Data Formatting
The design team had identified data formatting as the largest task associated with current mail merge systems. To address this, the design team added the concept of "data collections." It viewed data collections as a means for saving combinations of data modifications, fields, text, conditional statements, and groups of these elements for use in other documents based on the same information. The design team also expected conditionals to eliminate the need for much data formatting before creating a mail merge document.
However, a number of problems with the proposed data formatting system arose in interviews. First, the original system did not account for importing data from multiple sources. In revising the prototype, this problem was addressed by transforming the "Add Data..." function into an "Add More Data..." function after data had already been added to a data collection. Adding more data included an extra step in which the user was asked to choose fields with which to link the databases together before selecting new fields to add to the data collection.
After this revision, however, another two problems with data formatting were discovered. In the first, the user wished to turn a single field record into multiple records, as a single record contained multiple first names and last names. The team realized that it will need to add functions for copying, cutting, and pasting, but has not yet determined whether this should be addressed with a more sophisticated tool.
In the same interview, the user also encountered a problem when trying to add a field indicating whether or not records belonged to a specific group. The user intended to add this field based on the records' inclusions in a separate database of only that group. The "add more data" function was not useful in this case, because the intended field was implied rather than specified explicitly. The design team discussed inclusion of a special tool for inserting implied data, but it will have to determine whether this functionality adds too much complexity relative to its added benefit.
Confusing Labeling and Layout
The labeling of data collections, associated word documents, and some output options were confusing to users. At first, users were confused by the design team's replacement of a data source with a data collection. Recognizing that selecting a data collection is almost always the first step in the mail merge process; the design team chose to bring up the "Modify Data Collection" dialog by default when the user launches the mail merge tool. This screen presents users with a more familiar "Add Data" button, in order to get them started quickly and smoothly. Since a new data collection is created and the current merge document is associated with it by default, this process should be simpler for novice users. In addition, the label "Associated Documents" in the "Modify Data Collection" dialog was confusing, and the design team will change it to "Associated Mail Merge Documents."
The output options were confusing to users, because of their organization and labeling. Sorting of the output determined how the document was previewed, but it was placed below the preview controls. The design team will switch this order to make the controls more intuitive. The label "Email All" created confusion for one user, who expected email mail merge to send an email with the merge document attached instead of as the message body. The design team will provide a better explanation of this functionality in the email options dialog box and through a tool-tip. The function of the "More Options" button was somewhat confusing, and it may be changed to "More Output Options" in order to better communicate its function.
Document vs. Project Based Concept Model
Word and other mail merge solutions that the design team has encountered all employ a concept model based on the idea of performing mail merge on a single document by itself. However, with the initial paper prototype, one user suggested the idea of performing mail merge on multiple files at once. This could be useful when different records should receive not just different text in their documents, but different documents all together. This led to the idea of project based mail merge.
After debating the idea of project based mail merge, the design team concluded that fully realizing this goal would complicate more common single document merges excessively. In addition, it would be unnatural when combined with the editing functions of Word, which are all based on a single document model. However, the design team did allow for the highly useful function of multiple document merge by making a less drastic concept model change.
In the paper prototype revision, the design team changed the concept model of a data collection being associated with documents to the opposite concept model of documents being associated with a data collection. This made output a function of the data collection instead of a function of the document. Therefore, any document associated with a data collection could be merged based on the fields of that data collection. However, this concept model still allowed for the standard practice of creating the output for one document at a time without requiring the user to be aware of the paradigm shift.
Intuitiveness of Fields, Conditions, and Previewing
Results of interviews were not all negative, however; they proved that the design team created an intuitive system for manipulating fields and conditions as well as for previewing output. The general flow of the designed mail merge toolbar seemed to work well for users, and they were able to quickly begin placing fields into the document. One user commented on the convenience of not having to open a separate window when inserting fields. In addition, the visual system that the design team created for handling conditionals proved intuitive. Despite not having used conditionals with mail merges in the past, test users found their use with fields straightforward once they had been introduced. The preview mode functions very similarly to the one provided by Word, and the users quickly adapted to its few differences, such as operating with a toggle button and being indexed by the sort fields instead of record number. The design team will maintain these intuitive elements of the application and attempt to build upon them when further modifying the design.
Questions Not Addressed by the Paper Prototype
Because all of the interactions were first time uses, the designers were unable to test how power users would take advantage of the system over an extended time. The system provides functionality for re-using components in a data collection by saving individual edits and introducing groups. Groups were never used in the interviews, because the users only performed a merge on one document. Despite this, the design team believes that groups would be useful upon extended use of the application and that their presence did not confuse users during the simpler scenarios performed in interviews.
By nature of the paper prototype medium, the design team was unable to test the intuitiveness of components' appearances on screen. This was especially relevant when one user did not initially realize that plain text could be inserted into conditional results. This would hopefully become more clear on screen based on the appearance of the box in which to enter conditional results. It would also be important for fields, conditions, and output actions to have a distinctively different appearance from other document elements which could only be approximated with the paper prototype.