top of page
cui_grid_banner.jpg
Microsoft_logo.jpg

Conversational UI for Cortana

The journey of evolving Microsoft's digital assistant into a more familiar conversational experience

Everybody knows how to chat.

Even if you lived under a rock in the middle of nowhere, you'd still probably understand how to texts and chat. It's one of the longest standing patterns in existence for digital communication.

So why aren't we using it for digital assistants? All they do is converse with you. Let's ease the load off our users and make voice easier to understand.

chat2.png

My role

In 2017 I was tasked with taking what we learned from our Skype Integration and exploring what Cortana might look like with a more familiar conversational UI. I was the sole designer documenting patterns, controls, and best practices.

What I did

  • Incubation

  • Interaction design

  • Product vision

  • Prototyping & motion studies

  • ​Research coordination

  • Documenting patterns and best practices

Reminder_OneShot_01.png

Leveraging what we've learned

There was already a strong foundation for us to leverage when it comes to having a "chat-like" conversation with a digital assistant. Our users found the chat experience in Skype to be very intuitive, but didn't want to take the time out to type. We felt this problem could be easily remedied by the inclusion of voice as the primary input modality.

To kick off this exercise, we engaged in concept value testing with a number of conversational models to gauge user interest - including our current experiences.

Chatting with Cortana in Skype

jopri_Cortana_03-13-17_New Mauve.png
Hero Cards using title.png
cortanawindows.png

Model A (left) was the favorite among all participants

Mapping use cases to our new conversational platform

One of those most laborious tasks of this process was taking all of our supported patterns and mapping them to our new framework in a way that made sense.

Skills Framework.png

Mapping answers and query results

Notes_Lists_Reminders.png

Mapping interaction flows for natural language tasks

MorningCard_withSpeaker.png
jopri_Cortana_03-13-17_Card_Morning.png
MorningCard_withSpeaker.png

Ensuring visual metaphors were consistent with voice experiences

Lock Copy 14.png
Lock Copy 13.png

Exploring how to make skills more discoverable

Prototyping & motion videos

It's one thing to map a bunch of skills on a diagram. It's another to experience them. A huge part of my job was to make things come to life in order for partners to understand our design intent.

Personal updates

Intent-driven skill discovery

Cold-start quick actions

Making it real

After presenting my explorations, I helped with the transition to our satellite teams to execute on the vision across Windows and mobile. This was truly a team-effort and took years to accomplish.

Windows

Mobile

98342454fedd31eecbe33ea73bac89ab.png
unnamed.jpg

Thanks!

John

voice_skills_grid.jpg

Productivity voice skills for Cortana

suggested_tasks_grid.jpg

Suggested tasks

in Outlook

device_suggestions.jpg

Smart replies & predictive assistance

bottom of page