The company was conducting a research process with elementary and high school teachers from across the US when I started. The teacher interview moderation plan was well defined, but the process for compiling data and feedback was unstructured and needed good methods for turning the raw data into insights that could be used for product development.

So I used some techniques I've used for other projects, mainly a system for analyzing and categorizing qualitative data, to help the product team actually make sense of the somewhat sizable amount of data being generated from teacher interviews. At the end, I was able to pull together some insights that answered questions and recommend frameworks that enable a quicker understanding of teachers' environments, workflows, and needs and provide support when making product design decisions.

One of the most common needs and features was being able to quickly set and manage timers while instructing in the classroom. For teachers, timers are essential to managing a classroom.

The timer design that was implemented in the alpha had a few problems that mostly stemmed from inadequate prototyping and testing of timers in some sort of simulation of the classroom. There were collisions with other voice commands and triggers and also visibility and discovery problems.

Those kinds of problems are usually a result of viewing static, non-interactive screens that cause product teams to either gloss-over important aspects of usage and/or make flawed assumptions.

I'm a designer that loves prototyping, so I coded prototypes for timer concepts in Unity to play back on large monitors where you could attempt to simulate the usage environment and user workflow. I felt this was important for this feature, timers, because there are a surprising amount of important micro-interactions that can go into making a Voice UI timer feature that works well for a busy teacher in a loud classroom.

The video below is an artifact I put together for review of two different timer concepts.

A very important element of a Voice UI is the attention system. To start, you need to be able to tell people when the assistant is listening for commands, processing a command, and giving an answer. And it likely needs to be done for multimodal contexts: visual LEDs on a device, visual graphics on a screen, audio, and haptic. Also, you need to consider if your user is doing something important, like driving, where too much stimuli can distract from that critical task.

Another very important feature of a VUI's attention system is the schema for how interruptions are handled. For a general example, how do you prioritize notifications while Siri or Alexa is delivering or managing output while you're driving? Should a new text message stop playback of your Spotify song, and, if yes, should it immediately read the message or ask if you want to hear it read back? Is asking to hear the read-out too much of a distraction for a driver?

So I worked on interruptions for Merlyn's attention system, and created a mapping for how up to four modes (device LED, monitor display, a Merlyn avatar, and device audio) would behave for a number of different interruption scenarios. For a small example, should a paused video automatically resume when a teacher interrupts playback with a question for Merlyn? And are there situations where the answer is different for that same scenario?
‍
The unique challenge with Merlyn was that there are situations where a teacher wants to give Merlyn commands without interrupting an output the system is currently managing. So the mappings and animations shown below were an essential tool used to help the product team make decisions about how Merlyn handles interruptions.