© 2019-2020 by John Price

Conversational UI for Cortana

The journey of evolving Microsoft's digital assistant into a more familiar conversational experience

Everybody knows how to chat.

Even people living under a rock in the middle of nowhere understand how to send texts and chat online. It's one of the longest standing patterns in existence for digital communication.

So why aren't we using it for digital assistants? All they do is converse with you. Let's ease the load off our users and make voice easier to understand.

My role

In 2017 I was tasked with taking what we learned from our Skype Integration and exploring what Cortana might look like with a more familiar conversational UI. I was the sole designer documenting patterns, controls, and best practices.

What I did

  • Incubation

  • Interaction design

  • Product vision

  • Prototyping & motion studies

  • ​Research coordination

  • Documenting patterns and best practices

Leveraging what we've learned

There was already a strong foundation for us to leverage when it comes to having a "chat-like" conversation with a digital assistant. Our users found the chat experience in Skype to be very intuitive, but didn't want to take the time out to type. We felt this problem could be easily remedied by the inclusion of voice as the primary input modality.

To kick off this exercise, we engaged in concept value testing with a number of conversational models to gauge user interest - including our current experiences.

Chatting with Cortana in Skype

Model A (left) was the favorite among all participants

Mapping use cases to our new conversational platform

One of those most laborious tasks of this process was taking all of our supported patterns and mapping them to our new framework in a way that made sense.

Mapping answers and query results

Mapping interaction flows for natural language tasks

Ensuring visual metaphors were consistent with voice experiences

Exploring how to make skills more discoverable

Prototyping & motion videos

It's one thing to map a bunch of skills on a diagram. It's another to experience them. A huge part of my job was to make things come to life in order for partners to understand our design intent.

Personal updates

Intent-driven skill discovery

Cold-start quick actions

Making it real

After presenting my explorations, I helped with the transition to our satellite teams to execute on the vision across Windows and mobile. This was truly a team-effort and took years to accomplish.

Windows

Mobile

Thanks!

John

Productivity voice skills for Cortana

voice_skills_grid.jpg

Smart replies & predictive assistance

device_suggestions.jpg