Shabbir Faizy
A Usability Testing of two Vaccine appointment scheduling flows
The Vaccine Department was facing low number of vaccination appointment in the already shipped product even though the Covid infection rate was constant in the state. So we needed to improve appointment making flow and make it user friendly and intuitive.
We came up with a new flow but needed a solid proof to choose it was the right one , hence A/B testing was conducted on both flows.
Overview

The covid vaccination department of a state was one of our client , and we were working on delivering the portal with appointments , scheduling , admin flow.
There was decision taken to redesign the appointment flow to make it more intuitive.
a new flow was made keeping UX in mind and was tested against the previous flow.
Sector
Health , Government
​
Challenge
To deliver an intuitive flow of the vaccine appointment process.​
​
My Role
Research
Project Time
2 weeks
​
Process
Usability Testing - A/B testing​
​
Tool used
Maze
The control flow - Original
We designate the control flow to the original version . This was the flow that was first delivered for Vaccine appointment scheduling
-
It was simple , with basic verbiage
-
Not much images or icons
-
A simple flow with selection rounded buttons keeping it one selection per page
​




Prototype
The Variant flow - Modified
We came up with a modified version of the Appointment scheduling flow , this time making use of images and reducing the number of pages
-
Use of icon to make it easier for user to understand the action available like schedule , reschedule etc.
-
including two selections in succession like vaccination type followed by number of doses
-
Next and Previous CTA to complete the step




Prototype
Why A/B Testing ?
A/B testing was needed here because we were ready with two designs that is made on different suggestions on team discussions , At this point it becomes important to choose the best one which is done by this test and it helping achieving following goals
-
solves visitor pain points.
-
Reduces bounce rate
-
achieve statistically significant improvements.
-
redesign the flow to increase future business gains.
Usability metrics
Before diving into the test it was important to consider what would be the usability metrics upon which we would rank the flow by giving priority to metrics. so i came up with following metrics
-
Task completion - this was the first priority as it was the whole motive of making the user easily complete the scheduling task
-
Average time - this was of second priority as we also wanted users to spent as less time as possible and making them focused in action of scheduling and not overwhelming them by other not so important task.
-
Misclick - this was the third priority of finding the unwanted clicks
Results
The test was conducted on the platform Maze.
We hired users on maze platform and provided them the flows with each user being tested for only one flow. Their responses were recorded and detailed scrutiny was done.
All users were given the task to complete the flow by giving them a scenario of assumptions that they are a normal citizen of state and needs to schedule its first covid vaccination dose
The control flow
All users completed the flow as it was simple and short , so other metrics were recorded
2.4 s
Average time spent on the flow
Average misclick rate of flow
The simple flow design of the control flow provided a 100% success rate with fairly less average time and misclick
All users completed the task with zero bounce off
OPTIMAL PATH ANALYSIS
Analyzing usability of each screen and providing usability score to each and the inferences
Screen 1

6 s
Average Duration
Average misclick rate
-
Major hotspot is seen at the language selection button signifying its importance
-
Users very easily found the SCHEDULE VACCINATION as the major action
-
Misclicks are basically with the users trying to explore the page before starting the mission because the action button has less focus.
Screen 2
1.8 s
Average Duration
Average misclick rate
-
Users easily found the covid selection button

Screen 3

1.8 s
Average Duration
Average misclick rate
-
First dose action was easily found in quick time
The Variant flow
Need to examine each screen’s misclick rate to ensure users are finding what they need.
2.7 s
Average time spent on the flow
Average misclick rate of flow
-
The average time spent on screens is good but more than control flow.
-
Zero bounce off rate with 100% completion success.
-
Misclick rates is comparatively higher and needs to be examined every screen for improvements.
OPTIMAL PATH ANALYSIS
Analyzing usability of each screen and providing usability score to each and the inferences
Screen 1

3.1 s
Average Duration
Average misclick rate
-
Users very easily found the main button associated with task without any misclicks.
-
The icons played an important role of maintaining the focus.
Screen 2
4.6s
Average Duration
Average misclick rate
-
Easy selection when three options are available

Screen 3

3.6 s
Average Duration
Average misclick rate
-
Misclicks were found with user confusing between clicking of first dose and second dose.
-
Reason being sudden release of information on the same page
-
Selection of dose should be made on a separate page to avoid overwhelming of users
User rating Comparison
Control flow
Variant flow
9.8
Avg
Did the experience meet your expectations ?
9.2
Avg
9.6
Avg
How did you find the ease of use ?
9
Avg
Learnings
-
Both the flows stood on 100% completion success
-
Control Flow A was more successful than the Variant flow B in terms of usability and intuitive nature.
-
However variant flow B landing page was much better in getting the task done with the icons present along with action
A Hybrid flow here can be a better option comprising of landing page with icons and separate page for each sub actions to avoid misclicks.
Proposed Hybrid flow
A Hybrid flow was suggested with icons for landing pages to keep focus and reduce average time of completion along separate page for each sub actions to avoid mis-clicks which was a major concern in the whole testing result




Prototype
This was a very unique experience among all A/B testing and Usability testing done in my research career because we were expecting a better flow to get from this result but we got a hybrid version of both the flows with all good components.
Thank you for reading