Project Background
This project was part of my masters program and involved research and development into voice technology, conversational UX design, prototyping for voice and voice usability testing.
A prototype was designed using Airbnb as a case study, with particular focus on their ‘Experiences’ product. The goal of the project was to design a product that allowed Airbnb users to learn about experiences for an upcoming trip and book them using voice technology, either with their smartphone or smart home device.
I completed the research, prototyping and testing for this project. Key takeaways from each section are highlighted below.
Research
Market Research
With over 250 million active monthly users in the United States alone, voice technology, either with smart phones such as Siri or smart home devices such as Amazon Alexa, are experiencing significant growth rates amongst users. Users value how this technology allows them to perform tasks on previous spent time. Tasks such as checking the weather for the day or getting the directions to their next meeting are being performed using voice commands. This technology gives users access without any interruption of their current activity.
Problems with Conversational Commerce
Purchasing via voice devices is not that common due the issues users face. Firstly, users of voice devices are not getting the information they require to complete a purchase. Privacy is also an issue when purchasing via voice. Users need to feel secure when purchasing via voice as they do not feel comfortable giving their credit card details to a voice device. Users need to already have an existing, trustworthy relationship with businesses before committing to purchasing via this new technology.
Conversational UX Design
IBM are one of the leaders in the field of conversational UX design and have created a set of principles and guidelines that VUX designers can follow when designing new voice applications. These set of principles are called the Natural Conversation Framework (NCF.). There are 4 categories to the NCF. 1. Interaction Model. 2. Common Activity Modules. 3. Navigation Method. 4. Sequence metrics. All 4 categories were used as a reference throughout this project.
Prototype
A prototype was developed using Adobe XD’s new voice prototype tool. The prototype was designed to develop and showcase the features the ‘Airbnb Conversate’ product would have. Below are a selection of the most interesting part of the product and a video of the full conversation can be viewed at the bottom of the page.
Topic Guidance
A critical step in achieving a positive user experience in voice products requires a technique called ‘topic guidance’. This occurs when a voice agent asks eliciting questions to get more information from the user. The more detail a user provides, allows for a narrower scope of potential answers, which will increase the likelihood of the agent providing the right information to progress the conversation positively. The image shows a conversation with the agent (text in white speech inputs) using the topic guidance technique.
Providing Content
Users require a variety of different content to make purchases online. This can vary from images of a product, refund policies, customer reviews and much more. A conversational agent needs to provide this information to the user throughout the conversation. In the image below you will see the voice agent sending the images of the chosen cooking class to the users Airbnb profile to allow the user look at the images of the cooking class before deciding to purchase.
Purchasing with Voice
The research suggests that making a payment, the final stage of the customer journey, is the most uncomfortable for users when using voice devices. This is due mainly to privacy concerns surrounding vocalising card details and having them recorded. In this image, the feature of making a payment using previously saved card details with Airbnb is shown. This makes payments via voice seamless and safe.
Complete Conversation
To experience the complete conversation and its additional features, the full prototype video is below.
Testing
Five participants were tested on the ‘Airbnb Conversate’ prototype, using a set a of metrics developed through the Natural Conversation Framework (NCF). A brief description of the metrics will first be highlighted followed by a discussion of the most important insights from the test
Measuring a Conversation
The following metrics were tested on the final prototype.
Number of Sequences: This is a measure of how many base sequences or adjacency pairs were initiated throughout a conversation. This is used to test the level of engagement of a conversation, but also as a foundational metric that serve the other two metrics.
Sequence Completion Rate: This is the percentage of the base initiated sequences that were completed by the agent or user. This is essentially the rate of successful sequences throughout the whole conversation. This figure can highlight the level of mutual understanding both parties had throughout the conversation.
Interactional Efficiency: This is a measure of how much extra work both parties needed to do in the conversation to understand the conversation and progress. It is calculated by counting all the expansions in order for either the agent or participant to clearly understand what was being asked or said.
Results
Purchasing with Airbnb Conversate
100% of participants made a purchase via voice. This was a very exciting and positive step forward for this prototype. This outcome highlights two significant points. The first, that voice interaction can provide sufficient information to make purchases and secondly, the hypothesis made in the research section, where users are more likely to make voices purchases from brands they have made previous purchases with, is proven.
High User Satisfaction
Airbnb Conversate scored 85 in the SUS test. The most common words used to describe the prototype in the product reaction card test were ‘easy to use’, ‘effortless’, and ‘efficient’. These results show strong potential for the adoption of a complimentary product like ‘Airbnb Conversate’, especially for the scenario that the prototype was tested under, where a user has a holiday already booked and wants to learn about things to do.
Final Thoughts
I was very happy with choosing conversational UX design as a project topic. It was very exciting to learn about a new technology and apply the knowledge gathered through the research stage to a real and relevant business case such as Airbnb. Identifying the key pain points of conversational commerce and designing an effective solution was a very satisfying conclusion to this project.