Consumer Drop-offs
Case Study | GoCart
Identified by our analytics on Amplitude, currently there is a 71.9% drop-off rate after users click the GoCart checkout button on Firehousesubs.com. To dig into why this percentage is so high, I led a series of user research initiatives.
1 product designer
1 product manager
ux research, ux/ui design
Background
There is a large % of people who are actively clicking the GoCart checkout button, but for unknown reasons, they are not following through with enrollment/GoCart checkout. We’re not sure why this is happening - is it the lag time between clicking the button and modal opening? Are we under-communicating with users? What's really going on?
Goals
For these sessions, I hoped to overall gain a better understanding of what's going on after people click the GoCart button at Firehouse. Specifically, find out what the reason is as to why people may click the button, then decide to abandon the process.
Additionally, I looked into obtaining feedback around:
-
What makes users want to click the GoCart checkout button?
-
If they do click, what are they expecting to see next?
-
-
How long does it take for the user to click the button and then decide to not progress forward? May be able to get through analytics
-
When users drop-off, is it because they are missing some type of expected information after clicking the button? What information is this?
-
What can we do (that we’re not doing/better) to ensure drop-off doesn’t happen?
UXR process
My full research process including the activities I conducted are highlighted below:
RESEARCH PLANNING:
✅ Creation of prototype
✅ Create test plan including script
✅ Create test plan dashboard
✅ Review plan with stakeholders
RESEARCH SETUP:
✅ Creation of test on UserTesting.com
✅ "Testing" the test
✅ Scheduling moderated sessions
CONDUCT & ANALYZE:
✅ Run moderated sessions
✅ Analyze results to uncover themes
✅ Create highlight reels
✅ Share findings & recommendations
Screener demographics
I wanted to make sure that I captured a wide enough range of demographics for this study, but let the screener to be generally more broad. I've created this screener for past tests and it yields good results:
-
User is between 18-45 years of age
-
User is located in the United States
-
User has made an online purchase in the past month
-
3 users will be on their computer
-
2 users will use a smartphone
Success metrics
I was hoping for specific metrics to determine if the testing sessions were successful. In general, some of the feedback measures I was hoping to get back are as followed:
- Data to allow us to understand the main reason(s) that are likely causing drop-offs that can be turned into a tangible list to act upon
- The ability to provide value to our merchants with real data and feedback
- Consensus/sentiments around when GoCart pops up as users think out loud
Test script
[Introduction]
[Starter Questions]
Here’s where we can try to get additional feedback related to our industry/product. We don’t necessarily need to ask these questions, but it may be helpful data to have in our back pocket.
-
Talk about your most recent good or bad payment experience you had online. What did liked/didn’t like?
Thank you for sharing that with me. Now we’ll begin with our test!
[Tasks/Test specifics]
Checking out a sub through Firehouse Subs
I’m going to send you a link to a website that I’d like you to open. Please share your screen so that I can follow along.
Link: Firehouse Subs - Largo Mall #744 - Largo, FL
Zip: 33771
[Scenario] Let’s pretend you want to order a sub sandwich online for pickup from Firehouse Subs Largo Mall location.
Task: Go ahead and a sub to your cart and go through the process of checking out as if you were doing this in real life. For any personal details, feel free to enter your credentials as Usertesting.com will block your info, or feel free to make up any names and emails. We won’t be following through to actually purchase the sub so you can pause before entering adding any payment details.
-
If they clicked on credit card, pause and ask them why. Then steer them back to the main checkout.
-
Before clicking on GoCart Btn, ask what they would expect to happen if they clicked on that button. What would you want to happen? What information would you want to see here?
[Task] Now, let’s move forward with checking out with GoCart. Pausing here, please think out loud about what you’re seeing.
[Follow up questions]
-
What is this screen communicating to you?
-
Do you have enough information to proceed?
-
Do you think you would proceed at this point in a more real-life scenario? Why or why not?
-
What information would you want to see here?
[Question] What do you think happens next? What would you want to happen next?
Scenario: We’re not going to follow through completely with this order today, but at this point, you’d be able enter your payment details and then be able to pay.
[END OF TEST]
Thank you so much for answering my questions and completing the tasks. Now that we’re done with the test, I’d like to ask you a few post-test questions.
-
Do you feel like you have an understanding of what GoCart does?
-
Would you use GoCart if it was available to you at a Merchant you were shopping at or purchasing from in real life? Why or why not?
-
Did you feel as though you had enough information to proceed with using GoCart?
Findings
Main Themes
👀 Users who click on GoCart want to just take a quick look to see what it is.
😑 Once they interact with the button, they see an email to sign up and determine that's more work than just checking out as a guest.
🤝 Trust is important. Users want to trust GoCart from the start - before they click on the button and throughout the payment process.
😍 Value. Users want to know what value GoCart provides them beyond their current experience.
(5/5 users expected to see more info about GoCart)
Memorable Payment Experiences
Before they began the test, I asked users to talk out loud about their most memorable recent payment experience in efforts to understand even more what users liked or disliked about other competitors/non-competitors. If we heard repetition in this answer around something that GoCart doesn't do today, I wanted to capture it to take back to the product team for discussion.
What users like the most about their good payment experiences:
-
It was fast
-
Made a payment quickly
-
Efficient
-
“Didn’t have to think twice because info was saved”
-
2/5 users liked the rewards/enticing features, such as using a discount or having a promo available
Natural User Path
What is the natural path users take while trying to pay for their sub sandwich?
-
4/5 (80%) of users went with the merchant checkout option
View highlight reel: Natural User Path
What made users NOT click the GoCart button?
-
Most (4/5) users gravitated towards what they are most familiar with using, which was the merchant Credit option
-
Users had a sense of discomfort when it came to straying away from what is more familiar
View highlight reel: Natural User Path - Why?
What would make users more comfortable to proceed with the GoCart option?
-
Users in general want more information (trust!)
-
They want to know: what is GoCart? How can they get more information about GoCart?
-
Users are curious about GoCart’s value, specifically where else can they use GoCart?
What benefits do they get beyond this experience?
Other Insights
What additional information do users need (if any) after clicking the GoCart butt
-
4/5 users mentioned wanting more information on this modal
-
What IS GoCart?
-
The value of GoCart
-
Where else can I use this?
-
Is this a faster means of checking out?
-
Why should I sign up here rather than Firehousesubs.com?
-
-
2 users googled GoCart when landing on the first modal
-
1 googled it to get more information about it
-
1 googled it in hopes of finding out where else she can use it and ended up on the Merchant tab on the website
-
View highlight reel: Interacting w/ GoCart Modal
Suggestions & Recommendations
After 5 moderated test sessions, I felt like there was enough information gathered for a more clear path on moving forward. I suggested the following to the team:
-
Iterate design options based on feedback, specifically around giving users MORE information about GoCart on the first introduction to it
-
Run a series of unmoderated tests against designs focusing on whether the content compels users to take action/continue and message comprehension
-
Testing will provide us with a deeper understanding on how users react and respond to designs under consideration
-
-
Getting in the habit of pro-actively testing will allow us to continuously gain regular insights into potential users
Unmoderated Testing Series
As a product designer, I worked with the insights I had from my research studies to kick off a second round of testing with different prototype designs. The designs focused on what we can communicate to users, and why they should trust GoCart.
I created 3 designs to get feedback on from users that highlighted 3 different paths:
1. Value props of GoCart/Why users should join
2. What will happen next when users do continue
3. An ability for users to "Learn More" about GoCart without leaving the experience
Conducting the study
I set up 15 unmoderated usability tests using the same demographic screener as I did with the moderated tests.
-
5 users would be presented with the Value props prototype
-
5 users would be presented with the What happens next prototype
-
5 users would be presented with the Learn More prototype
As a product designer, I worked with the insights I had from my research studies to kick off a second round of testing with different prototype designs. The designs focused on what we can communicate to users, and why they should trust GoCart.
I created 3 designs to get feedback on from users that highlighted 3 different main focal points for GoCart:
1. Value props of GoCart/Why users should join
2. What will happen next when users do continue
3. An ability for users to "Learn More" about GoCart without leaving the experience
Conducting the study
I set up 15 unmoderated usability tests using the same demographic screener as I did with the moderated tests.
-
5 users would be presented with the Value props prototype
-
5 users would be presented with the What happens next prototype
-
5 users would be presented with the Learn More prototype
Questions for unmoderated testing
-
[Verbal Response] Take as much time as you need to review the content presented. What, if anything, draws your attention or stands out to you? Please share your thoughts aloud.
-
[Verbal Response] If you had a magic wand, how would you improve this content?
-
[Verbal Response] What do you want to know more about?
-
[Rating Scale | Strongly disagree - Strongly agree] Please rate your level of agreement with the following statement: "This content compels me to take action." Please explain your rating.
-
[Rating Scale | Strongly disagree - Strongly agree] Please rate your level of agreement with the following statement: "I feel like I have an adequate amount of information." Please explain your rating.
Prototype Options
Value Props
What Happens Next?
Learn More
Findings
Main Themes
🙇♂️ Users have a baseline understanding/assumption of what GoCart is
This could be due to:
1. The Proximity Design Principle (seeing GoCart in context alongside other checkout options)
2. Jakob’s Law of Internet User Experience (users are familiar with other apps they use more often)
👀 Users who want more information will go seek it. There's also a fine balance between the amount of information being displayed and an innate curiosity of continuing
✅ User's feel that "log in" and "join now" should be treated as two separate paths and actions
❓ Users are still inquiring on where GoCart can be used
Theme 1
Users who want more information will go and seek it
In versions Value props & What happens next? users wanted more information and sought out ways to get it.
Example: clicking on prototype to expose more info, trying to type their emails in, googling, etc.
Supporting Quotes
“Maybe a hyperlink to see more information.”
“I would want another window to read more about gocart instead of just signing up and going immediately.”
“Maybe a "explore more about gocart option"”
“It would be useful to have maybe a "new to gocart? click here to get to know us" link to website”
“Link to gocarts website so I can learn more about it if using for the first time”
But we know that users don't read...Users who want more information will go and seek it
In versions Value props & What happens next? users wanted more information and sought out ways to get it.
Example: clicking on prototype to expose more info, trying to type their emails in, googling, etc.
Supporting Quotes
“Maybe a hyperlink to see more information.”
“I would want another window to read more about gocart instead of just signing up and going immediately.”
“Maybe a "explore more about gocart option"”
“It would be useful to have maybe a "new to gocart? click here to get to know us" link to website”
“Link to gocarts website so I can learn more about it if using for the first time”
Theme 1
Users feel that "logging in" and "creating an account" are two separate actions
1/3 of users brought up confused sentiments around the one field for email being the mechanism to both sign in and create an account.
Supporting Quotes
“So this is both a sign in and a sign up window. Interesting...”
“It would be useful to have maybe a "new to gocart? Click here.”
“I feel like log in or join now should be separate - I could be confused.”
“Seems like I can continue or actually could create an account. But how do I create an account here? I would prefer a clarified way like sign up here.”
“I would feel better if there was a button to allow me to sign up rather than asking me for an email because I don't have an account to give an email address.”
“Is the one time code to create an account at or is it a one time code to process payment?”
“I think the part that really confuses me is how it says log or join now and there's no other button down here. I feel like I'd feel this info in and then click log in, or click a join now and take me to the info I'd need to fill out.”
Information Rating
"I feel that I am given enough information to continue."
Version Value Props:
Score 4.3
Version What Happens Next:
Score 3.4
Version Learn More:
Score 4.3
What this means:
In order to make a decision on whether to continue or not, users prefer a mix of both what GoCart is and how it can be used.
Suggestions & Recommendations
I combined the findings from this study with the first moderated ones to come up with an actionable list of suggestions that we know will provide real user impact.
- Exploration around ‘Learn more’ link on current modal
This could be a quick win for both the users and also allow us to add a new component to our design system (the Learn More overlay)
- Research Try out a more traditional way of “Sign in” and “Sign up”
Addresses user confusion around what path they should proceed with
- Bringing in “In your own words..” question into consumer survey
I created a consumer survey that was sent out to real GoCart users and added this question to it in hopes of understanding even further how real users communicate/understand what GoCart is
- The Learn More option provides us with the most balance of information and flexibility.
There’s room for exploration on copy for the first screen - we’ll work with UX writer
How can we bring in more content around where GoCart can be used in “Learn More?”