Typetalk, a Business Chat App: Prototype / Usability Testing
Background
Typetalk is Nulab’s business chat app, designed to replace email and other messaging software. Colleagues discuss projects and other work-related topics through Typetalk’s chat rooms and direct messages.
This project followed an initial study that revealed navigation issues with Typetalk’s “chat room” list and tested a prototype designed to relieve those issues.
The Problem
In the last study, participants had difficulty navigating to their desired “chat room”. The chat room list was frequently reordered, as it was designed to sequence chat rooms according to last activity. As a result, it was difficult for users to predict where a desired chat room would appear on the list and forced users to do a tedious search of the list each time they needed to communicate in a chat room.
The Prototype
2 solutions were tested for this study. The first was a “pin” feature that allowed users to “pin” a topic to the top of the chat room list, allowing them to prioritize specific chat rooms for easy navigation. The second was a “grouping” feature that allowed users to group multiple chat rooms, providing a logical, user-organized chat room list.
Project Goals
The goals of this project fit the goals of a typical prototype testing project, testing if its solutions address user challenges and could be engaged with easily.
Evaluate the prototype’s effectiveness in helping users navigate the “chat room” list.
Evaluate participants’ ability to utilize the feature.
Identify usability issues.
Participant Recruitment
Recruitment for this project posed a minor challenge. We wanted feedback from both existing and prospective customers. However, Typetalk is Nulab’s newest product and didn’t have an easily accessible pool of customers from which to recruit.
After some discussion, stakeholders and I concluded that Nulab employees could be leveraged. Nulab employees were screened along behavioral characteristics to ensure we received representative feedback.
Experience level w/ team chat software
Frequency of use w/ team chat software
Scenarios & tasks that prompt use of team chat software
Fielding Research
In preparation for research day, I worked with stakeholders to design the discussion format below and drafted a corresponding script.
I tested the prototype with 7 participants, in 30 minute testing sessions. Interviews were conducted in-person. Testing sessions would focus on participant’s ability to engage with the new features.
Following testing, participants would be asked to continue engaging with the prototypes for the next 2 days, then asked to complete a survey, measuring the impact of the new features across daily use.
Usability Testing: Understanding Current Usability Issues
Participants tested the Typetalk App by attempting to execute the tasks below. Observing their interactions gave us an in-depth look at how the app performed.
“Pin” Feature
“Pin” Chat Rooms
“Unpin” a chat room
Find and navigate to pinned, chat rooms
“Topic Group” Feature
Create a “Group”
Add chat rooms to a “Group”
Remove chat rooms from a “Group”
Delete a “Group”
Find and navigate to grouped, chat rooms
Follow-up Survey
How has the “pin” feature impacted your navigation to chat rooms?
It’s made navigation slower/more difficult.
It’s had no impact.
It’s made navigation faster/easier.
Please Explain.
How has the “topic group” feature impacted your navigation to chat rooms?
It’s made navigation slower/more difficult.
It’s had no impact.
It’s made navigation faster/easier.
Please Explain.
Stakeholder Alignment
While research was conducted, stakeholders observed live or at their discretion via recorded sessions. All stakeholders recorded their observations and shared them later in facilitated, affinity mapping activities.
As facilitator, I guided affinity mapping, helping to align stakeholders on what was learned about how the prototype features performed.
Key Findings: Fast & Easy Navigation
Overall, the new prototype features were welcomed changes to the product. All participating in the study realized large improvements to their navigation experience. Once commonly navigated chat rooms were pinned or grouped, participants had virtually no issues finding and navigating to them.
Key Findings: Adding Chat Rooms to Topic Groups
Participants did however encounter friction in adding topics to topic groups. Many expressed it as a tedious process that could be made easier by drag-and-drop interaction, where users could drag a topic into a group, instead of following a series of tedious steps.
Summary & Deliverables
By the end of this project, we were able to meet the goals set forth at the beginning of this project of:
Evaluate the prototype’s effectiveness in helping users navigate the “chat room” list.
Evaluate participants’ ability to utilize the feature.
Identify usability issues.
Tangible deliverables that were provided at the conclusion of this research included:
Topline summary describing key usability findings.
Next Steps
Following this study, stakeholders would take the study findings and evolve the prototype to incorporate the easier drag-and-drop interaction.