As a solo UX practitioner, I’ve unfortunately had no one to rely upon but myself. While I can mitigate the pains of self-reliance during various design phases by choosing talented companions from the companies I’ve worked for, user testing has always been a one-man job.
A few of my colleagues I’ve talked to about this mentioned that they started hiring interns to take notes for them. While seeming like a good idea, there are a few drawbacks to this method:
1. User Annoyance
On one-on-one testing, users are already terrified. The way user testing is done, they are usually crammed into a small room in an unfamiliar building. I tend to request a large and empty-ish area for user testing, but even UX studios with dedicated user testing rooms do not take the claustrophobic sensations into account, so it might not always be possible.
Having a third person in the room, on top of this uncomfortable situation, is likely to tense up the participant even more and increase the amount of incorrect and unreliable data they provide. They already feel like they have to please you with their answers, having another person in the room will increase that sensation exponentially.
2. Information Prioritization
If you are already recording data, screen or sound, you don’t need to write everything down. You already have whatever you need to see in front of you.
What you observe, on the other hand, is a completely different story. You’ve been to theatre plays, concerts, any live event; you know that recordings don’t do them justice. Some nuances can only be experienced while you are there, in person.
While user testing, you will want to catch those small details. If your intern or colleague can catch everything you’ve caught, great! But they usually cannot. Instead, they will end up writing everything down, which is unnecessary, since that everything is already on tape. What you’ll end up with is twice the data, and none of the details.
Some controlled distraction is good. Whatever you’re testing probably won’t be used in a distraction-free environment, because there is no such thing left; the closest I get is the Starbucks I’m typing this post in, and even then I’m distracted by doors opening, milk steaming and espresso pouring.
I even try to incorporate some controlled distraction into my tests by having employees walk in once or so to the room, to ask if the participant wants anything to drink if nothing else, and to see if they can quickly pick up where they left off.
However, having a secondary constant distraction, besides me -the moderator- is definitely not a desired effect on the test. Instead of focusing on two things, the participant will have to focus on 3 things, even if involuntarily. No good can come out of it.
How to take notes
What I Do: Use a proper testing sheet
I’ve designed a minimal user testing sheet for myself. I keep improving it with every test, but I’m quite happy with what I’ve got so far.
Write the name.
Have an area for your users name. I forget names the moment they are uttered. So, I always write their names at the top of the page. Any other ethnographic data required for the test can also be appended to it.
Know which task is answered
You also want to have note taking areas specific to your questions. That way, you can put symbols next to your scenarios for failed/successful tasks, and have task-specific reminders of your research. No more than 5–6 questions though. Don’t try to learn everything, focus your research on specific areas for more quality results.
Mark the time
Have a time-sheet, especially if your recording system doesn’t have a remote marking device. 5 minute intervals of up to 30 minutes does the trick for me; I just mark closer to where it is, and make a short note of whatever it is that intrigued me. That way, I will know to concentrate on that specific moment on tape when reviewing them later.
Be prepared to be surprised
Finally, an extra notes area does not hurt, for anything that does not fit to questions or the timeline. Something told seemingly out of context might actually be really important while preparing the report. You also might run out of space to write, even while taking short notes, so having an extra area will be useful. (You still should have empty papers with you, just in case. You might run into a fascinating participant once in a blue moon.)
Using a fountain pen while writing is also the most convenient way of taking notes. You don’t have to push on the paper at all, so you can comfortably and effortlessly take notes while listening to you user. My favourites are Lamy Safari, TWSBI Eco and Pilot Metropolitan, but your mileage may vary.
What I Wish To Do: Shorthand Writing
Honestly, shorthand writing seems like the next logical step. The latin alphabet is awesome to read and decipher, but is not handwriting friendly. News reporters have used shorthand for a long time for a reason; it’s quick and convenient.
Note taking can affect user behaviour. Write too much, the conversation will falter and you'll lose contact. Write too little, they'll think they're not worthy. They will also want to catch a glimpse of what you're writing about them, even though it is not actually about them.
My theory is that, shorthand will fix a few problems at once; You won't lose precious time to your scribbles, your users can't read what you're writing, and even if there is nothing note-worthy happening you can just scribble nonsense to make it look like you're interested.
I plan on starting with the shorthand alphabet, and then moving on to word-based alternatives. The learning curve is steep, but I feel that the benefits will be plenty.
Download the Note Sheet
You can download the note sheet photographed above, in Sketch format. Let me know of any improvements you've thought of.
Get Help With User Testing
Would you like help with user testing for your app or service? Get in touch and let's have a chat about how I can help you.