Usability Testing
Discussion of Interviews
We have broken down the discussion of what we've learned into sections that reflect different parts of our interface. Our discussion and conclusions for each section draw from all three of our interviews. Overall, we have noticed that our biggest pitfall has consistently been with using terminology that makes sense. We spent a lot of time deciding the diction only to have our users approach our terms with confusion. This has been consistent throughout our design process, and affects every section listed here. On the positive side, our visual placement and flow aided us immensely. When users could not understand what to do because of terminology, they almost always found their way just based on the spatial placement of our buttons. We've learned a lot from this phase and plan on making some big improvements.
Data Collection
Users had a difficult time figuring out what a "Data Collection" was. The terms "Associated Documents" and "Manage Data" were even more troublesome for understanding. Users thought that double-clickin an associated document would open up a new document with which to work. The hardest thing for each user to do on this screen was to import data, which is currently done with the "Add Fields..." button. The wording on this button did not make it clear that this is the method to import a database. We plan on revising the layout of this screen and the terminology to better reflect our user's concept model. If the "Directions" button (which should be renamed to "Step by Step Directions") were active, it might also help alleviate some confusion by explaining what a data collection is before users are introduced to it.
Also, we think that creating a less permanent way to remove a record - like checkboxes instead of the "Remove Record" button - would be a good change.
Selecting fields went over very well, with the exception that one user wanted to reorder the fields on the "Select Fields" screen. We could implement this with little consequence and have the order reflected in the order that the fields appear on later screens.
To select the data collection from the main interface, we want to change the name of the data collection to a drop down combo box that has each of the data collections listed. The last one (even if the list is empty) will be "New..." and that will bring users to the "Data Collection" window. There, we can eliminate the "Data Collections" group and move the "Associated Documents" group to the advanced output screen, thus eliminating our need for the entire top half of the current screen, which should eliminate a huge amount of the confusion. When combined with rewording "Add Fields..." this screen should become much more intuitive.
Linking Data
Two of our three users thought that linking data was another way to choose which fields would be used. One even tried dragging a new field into the list of the old. This needs to be redescribed. But once the users understood what needed to happen, they often wanted to link every single field. Again, this needs to be better explained. This interaction should be visual as well as textual, but the terminiology needs to be better for this to be understood - we should make this less technical.
The bottom section of linking data (choosing the method for adding fields) seemed self-explanatory, and none of our users had a tough time understanding it.
One user pointed out that linking databases in this way requires knowledge in the head about each database. If we could implement a preview of the records attached to the fields, it would make linking much easier.
Document Type
The document type buttons generated a little confusion. We think that implementing changes to the format of the document based on the type (e.g. label boxes appear when "Labels" is selected) would rectify any problems.
One specific change for the "Email" button is to make the "From" field default to the current user of the local machine.
Fields
The fields interaction was extremely well received. It is arguably our most important interaction, and every user performed it intuitively and naturally without any prompting.
Conditions
Once understood, conditions were also well received. Their placement and concept required a little effort; however each of the users found it and understood it eventually. Some minor terminology and layout changes would enhance conditions: "Groups" should be renamed "Conditions and Groups" since it now contains both; the "New Group" and "New Condition" buttons' positions should be switched; and, within the condition, "True" and "False" should be changed to "yes" and "no" for a clearer user understanding.
Preview and Edit
Actually implementing this button would help increase user understanding, but as it is, none of our users really had any problems with the concept. The implementation just needs to be clear that once in "Preview and Edit" mode, the user can look at and change each individual record. One suggestion we may implement is the ability to print a single example document from the merge. This would allow the user to preview an example of the finished job.
Output
The output sorting on the main interface could also be a drop down combo box that had several options for choosing which field to sort by.
On the "Modify Output Sorting" screen "Add Field" button is very poorly worded. One option is to change it to "Then by...". This screen is also redundant, as it gets repeated in the "Advanced Output" screen. We want to have these two each reflect changes on the other.
The word "Output" itself for this section may not be the best word, however, thinking of something better would probably require a large time investment.
Overall, we had the most success with terminology on this screen. The word "Action" had no apparent cognitive friction, and "Do it!" was very well received. Users became energetic and excited on this screen when realizing everything they could do, finally pressing the "Do it!" button to complete the process. It was a good way to end the interviews.