top of page

Shabbir Faizy

A Usability Testing of two Vaccine appointment scheduling flows

The Vaccine Department was facing low number of vaccination appointment in the already shipped product even though the Covid infection rate was constant in the state. So we needed to improve appointment making flow and make it user friendly and intuitive.
We came up with a new flow but needed a solid proof to choose it was the right one , hence A/B testing was conducted on both flows.

Overview

The covid vaccination department of a state was one of our client , and we were working on delivering the portal with appointments , scheduling , admin flow.

There was decision taken to redesign the appointment flow to make it more intuitive. 

a new flow was made keeping UX in mind and was tested against the previous flow.

Sector

Health , Government

​

Challenge

To deliver an intuitive flow of the vaccine appointment process.​

​

My Role

Research

 

Project Time

2 weeks

​

Process

Usability Testing - A/B testing​

​

Tool used

Maze

The control flow  - Original

We designate the control flow to the original version . This was the flow that was first delivered for Vaccine appointment scheduling

  • It was simple , with basic verbiage

  • Not much images or icons

  • A simple flow with selection rounded buttons keeping it one selection per page 

​

Prototype

The Variant flow - Modified

We came up with a modified version of the Appointment scheduling flow , this time making use of images and reducing the number of pages

  • Use of icon to make it easier for user to understand the action available like schedule , reschedule etc.

  • including two selections in succession like vaccination type followed by number of doses

  • Next and Previous CTA to complete the step

Prototype

Why A/B Testing ?

A/B testing was needed here because we were ready with two designs that is made on different suggestions on team discussions , At this point it becomes important to choose the best one which is done by this test and it helping achieving following goals

  • solves visitor pain points.

  • Reduces bounce rate 

  • achieve statistically significant improvements.

  • redesign the flow to increase future business gains.

Usability metrics 

Before diving into the test it was important to consider what would be the usability metrics upon which we would rank the flow by giving priority to metrics. so i came up with following metrics

  • Task completion - this was the first priority as it was the whole motive of making the user easily complete the scheduling task

  • Average time - this was of second priority as we also wanted users to spent as less time as possible and making them focused in action of scheduling and not overwhelming them by other not so important task.

  • Misclick - this was the third priority of finding the unwanted clicks 

Results

The test was conducted on the platform Maze.

We hired users on maze platform and provided them the flows with each user being tested for only one flow. Their responses were recorded and detailed scrutiny was done.

All users were given the task to complete the flow by giving them a scenario of assumptions that they are a normal citizen of state and needs to schedule its first covid vaccination dose

The control flow

All users completed the flow as it was simple and short , so other metrics were recorded

2.4 s

Average time spent on the flow

Average misclick rate of flow

The simple flow design of the control flow provided a 100% success rate with fairly less average time and misclick

All users completed the task with zero bounce off

OPTIMAL PATH ANALYSIS

Analyzing usability of each screen and providing usability score to each and the inferences  

Screen 1

6 s

Average Duration

Average misclick rate

  • Major hotspot is seen at the language selection button signifying its importance

  • Users very easily found the SCHEDULE VACCINATION as the major action

  • Misclicks are basically with the users trying to explore the page before starting the mission because the action button has less focus.

Screen 2

1.8 s

Average Duration

Average misclick rate

  • Users easily found the covid selection button

Screen 3

1.8 s

Average Duration

Average misclick rate

  • First dose action was easily found in quick time

The Variant flow

Need to examine each screen’s misclick rate to ensure users are finding what they need.

2.7 s

Average time spent on the flow

Average misclick rate of flow

  • The average time spent on screens is good but more than control flow.

  • Zero bounce off rate with 100% completion success.

  • Misclick rates is comparatively higher and needs to be examined every screen for improvements.

OPTIMAL PATH ANALYSIS

Analyzing usability of each screen and providing usability score to each and the inferences  

Screen 1

3.1 s

Average Duration

Average misclick rate

  • Users very easily found the main button associated with task without any misclicks.

  • The icons played an important role of maintaining the focus.

Screen 2

4.6s

Average Duration

Average misclick rate

  • Easy selection when three options are available

Screen 3

3.6 s

Average Duration

Average misclick rate

  • Misclicks were found with user confusing between clicking of first dose and second dose.

  • Reason being sudden release of information on the same page

  • Selection of dose should be made on a separate page to avoid overwhelming of users

User rating Comparison

Control flow

Variant flow

9.8 

Avg

Did the experience meet your expectations ?

9.2 

Avg

9.6 

Avg

How did you find the ease of use ?

Avg

Learnings

  • Both the flows stood on 100% completion success

  • Control Flow A was more successful than the Variant flow B in terms of usability and intuitive nature.

  • However variant flow B landing page was much better in getting the task done with the icons present along with action

A Hybrid flow here can be a better option comprising of landing page with icons and separate page for each sub actions to avoid misclicks.

Proposed Hybrid flow

A Hybrid flow was suggested with icons for landing pages to keep focus and reduce average time of completion along separate page for each sub actions to avoid mis-clicks which was a major concern in the whole testing result

Prototype

This was a very unique experience among all A/B testing and Usability testing done in my research career because we were expecting a better flow to get from this result but we got a hybrid version of both the flows with all good components.

Thank you for reading

bottom of page