Content is part of your experience, and other lessons from user testing

On the Storytelling Studio, we constantly iterate on process as its relates to both how we work and what we work on. This includes figuring out the most effective way to test usability of our projects. For our most recent collaboration, Curbed’s 10 streets that define America, we had the opportunity to test our assumptions with nearly final content.

Test your assumptions

When designing, building, or writing stories, it’s easy to make assumptions about what users will want in an experience. User testing is an important step to evaluate those assumptions with real users, determine the usability of your work, and identify areas for improvement. By testing these assumptions early and often, you can refine your work to create a better experience for your audience.

What you can learn from user testing

Depending on the scope and number of tests you complete, you can learn actionable information from user testing. You can test users interest in a topic, their comfort level when exploring an experience, and whether user interface elements are clear and communicate what you intended them to.

For Curbed’s 10 streets that define America, we set out to test a few things:

• Was the navigation clear? Can users move between pieces of the story if they want to?
• Do users notice the personalized data element? Were there any takeaways from watching users interact with this?
• Did the chapter names encourage users to read that section?
• Was the introduction compelling enough to get users invested in the experience and encourage them to dive deep into the content?

User test your content, not just your design

In previous projects, we often user tested before receiving written and visual content that was close to final — it was a way for us to make progress on design while photographers worked on finalizing images and reporters finalized words.

However, we’ve found that user testing stories is far more successful if you test with content that is close to final. By testing at this stage, you are able to receive feedback on the overall experience of a story, which includes words, photographs, design elements, and sometimes more.

When user testing, we’ve found the most success assigning one person to interact with the user and one person to take notes of all the communication. We also record every session for our own records, and refer back to them if comments were missed by the notetaker.

For Curbed’s story, we user tested with an interactive prototype created in Marvel, and asked users to:

• Identify what they do first upon landing on the introduction
• Describe what each section is about before clicking through to the internal pages
• Enter one of the sections and find the second city within that section
• Explain how they would discover other cities within the story
• Outline why they were compelled to interact with the data (if they did)
• Talk through what their feelings were about customizable data (if they didn’t interact)
• Discuss their main takeaway from the story
• Summarize the story in one sentence

Take action on your findings

After user testing, it’s important to consolidate your findings in order to prioritize changes that need to be made. On the Storytelling Studio, we organize high-level takeaways in a document, providing any necessary context. From there, we outline potential changes to the project that will help address the takeaways. This document is then presented to the entire team, including reporters, social editors, project managers, and developers, and we agree on the changes we’ll implement in order to create a better experience.

For Curbed’s story, here are a few of the updates we made to our work after user testing:

• Reduced the length of the introduction, to enable users to dive into the main content quicker
• Included a number for each section to indicate each section is part of a whole
• Included the page you are on in the top-level navigation
• When opening the navigation menu, indicated what page you are reading

The most important part of this step is open and honest communication. By presenting the findings in an organized way with room for conversation, everyone on the team felt comfortable making changes to the project based on what made the most sense for our audience.

Depending on your projects timeline, user testing can happen in multiple rounds. After making considerable updates to improve usability, it’s good to start at the beginning to test your newest assumptions.