At the beginning of 2019, we had almost finished the design of our Customer Service Saas solution for Zappos.
Read more: How we used a Design Sprint to create a Customer Service Saas solution
We were at a point where we needed user guidance to make sure we were on the right path, especially around some long-term patterns we were introducing in the tool.
I quickly created a protocol to go and test our designs with the users. Our ops team helped with the scheduling, and we were ready to start!
Method
Schedule meetings with 5 Customer Service members of varying experience who have some sort of familiarity with our Customer Service Product, for 1/2 hour interviews.
In an interview setting, ask the interviewees to evaluate the concepts presented to them.
Right after that, ask them to perform tasks with the given interface.
Our line-up is following these guidelines:
- one facilitator: run the session
- one note-taker: document the session for both conversational and behavioral takeaways
- one observer (non-essential): this is a learning position for Research mentee
- the observer should facilitate the next session
- the facilitator is the only speaker.
- the facilitator will open the forum for questions halfway through and at the end
- if the note taker and observer want to ask a question outside of the above forum, they should signal it to the facilitator, who will introduce the question so not to overwhelm the participant
Housekeeping
- Remind the participant not to share any details outside of the test, as to not spread bias
- Remember to ask tenure at Zappos and previous Customer Service experience
- Ask if they are willing to come back to test again in the future
- Try not to book rooms with Solstice, because of the connection difficulties and latency
Prototype to test
InVision link: [withheld]
Concept Test Primary Goals
- Understand how the current Customer Service tool Home is valuable to agents, and why
- Introduce the new concept for the Customer Service Platform (CSP) header interface and identify the action areas
- Understand whether users find tabs valuable
Concepts to test
- Do they understand the options in the header?
- Do we need to spell it out or are icons enough?
- What essential options are we missing?
- Are the Status Popover and Notification badge meaningful to them?
- Can they actually see it?
- Is the tab-based structure of Contact handling valuable?
Success Criteria
- They understand the header structure and can tell which option is which
- They find the workflow broken into tabs valuable
Questions
- Let’s assume this is the Homepage of CSP in the future, which is the first thing you see after you login and after you close a Contact. What would you like to see in it?
- How is the current Customer Service tool Homepage useful to you?
- Look at the header: what options do you think we have up there? Is there anything missing?
- Tabs: how often do you have to open new tabs in your current workflow?
- What do you think about this breakdown (point at screen)?
Usability Test Primary Goals
- Can agents claim an incoming email?
- Can agents navigate through the entire flow easily?
- When guided, would have they found the options otherwise easy to access?
Usability Test Secondary Goals
- Can the user find and explain secondary features?
Success Criteria
- Agents can claim an email
- Agents can compos a response
- include adding a blurb
- Agents can fix issues and review the response
- Agents can wrap-up the contact
Scenarios to test
- A customer is sending an email about asking for a coupon. They are a popular Youtuber and want to be sponsored by Zappos.
- Release promo and reply to customer
Questions to ask
- How easy was going through the workflows?
- What did you find the most painful?
- What did you find hard to understand and how much training you think you would need to operate this new Phase of CSP?
- How would you change what you just went through?
Limitations
- We are testing a happy path. If this is clear, we’ll add edge cases
- Limited click path in InVision
- Designed with our new Design System, which is a brand new look and feel
- 30 minutes to ask questions and go through the tests
- Scheduled participants come from various Customer Service teams, and have different backgrounds and level of knowledge
