Account Payments
How can we help customers be aware their account is due?
Speak to staff, interview customers and source data. Add questions to Rolling Study/Continuous Discovery.
Workshop, Ideate and design with team. Check feasibility with other departments, stakeholders and engineering.
How does this align with other work currently happening in squad (dashboarad design, language, etc Are other squads conductiong research that can be leveraged.
Run an experiement, write lean canvas. Define test participants and measure success criteria.
Affinity mapping Miro board
Product Manager organised a team session in Miro where we asked all our questions and thought about all the things we knew, what we didnt know, some possible solutions and where we could focus our attention.
Rephrase the problem
Scenario
As a credit customer, how can Reece make me aware that I have money owing on my account?
Utilise our teams experience and brainstorm some ideas.
Traffic light system of colours
Dashboard - Running timeline of when bill is due
Something small
Actionable from the desktop
A timeline (horizontal)
A dial of sorts
Add to calendar
Ability to action from homescreen (Pay now)
People probs place more orders than pay invoices
Better clarity around whats due now, whats due next month
Widget - they can put on their phone like optus (can check at any time)
Onbarding - One of the tiles explaing the benefits of max can include prefs about alerts
Notification options, Push notifications, SMS, email
Maybe default to one unless they change?
Affinity mapping Miro board
Utilise insight gathered from the Rolling Study.
Add questions to the Rolling Study specifically around payment reminders,
how do our customers currently respond, what would help them, their thought processes, etc
“If I got a notification from the app, I don’t know how they do it add the link it to your calendar
or something and that all works like that and just the option to do that or even an option
through the app to say notify me the day it’s overdue or the day the months out or week before or whatever you know,
that’s be something I’d get around."
How often do customers go to AP dashboard?
Who is paying the account?
Size of business?
Payment method? (Branch, Quickpay, BPAY)
Not as much data as we would like but as a result we are adding more filters,
events and tracking more paths. Business categories, sizes, methods, etc
Customers are often confused by the language on the dashboard and pay the wrong amount.
Customers, when often prompted for payment, pay the full amount owing which may include 3 buckets: Overdue, Current and In progress. This is more work for internal teams.
When designing the experiment, we utilised the ideas from our workshop, customer interview data,
feedback from internal staff who are on calls to customers who need financial assistance.
We were designing for both web and native environments. For the experiment we implemented a "traffic light system" for alerts:
Blue: Ready for payment
Yellow: Due in 7 days
Red: Your account is overdue
Less than 1% of customers clicked "Never show me again"
Problem
Many customers are paying statements once they become overdue. Juggling multiple responsibilities,
lack of awareness and not knowing the the actual due date is the feedback that has been received.
Customers are currently notified twice a month. The 1st of the month when their statement is available for payment
and the 29th, when the statement is about to become overdue
Hypothesis
We believe customers lose track of when their account is due for payment
so if we surface payment notifications to the max dashboard
we will see an increase in customers paying their accounts on time
Overview
Customers who have permission to make payments in max will receive a
notification message on the dashboard when a statement is available for payment.
The notification will notify users the statement amount and give them the ability
to make a payment, snooze or delete the notification for good.
Design
The first experiment displays End of month (EOM) notification on the maX dashboard to users who have permission to Accounts & Quickpay.
Users have the option to click the "Pay now" button which takes them to the Payments dashboard or click the "x" with the ability to mute the notification for the current month or forever.
Launched August 30, 2022
Goal
The primary goal of the experiment is for users to see the payment reminder and click "Pay now"
The secondary measure is to monitor the number of people cancelling the notification forever or
if they disable the notification for the remainder of the month
Success criteria
5% of users click "Pay now"
Less than 2% of users cancel the notification forever.
EOM Payment Reminder Action Results
In the first 2 hours from activation of the experiment from 3 pm - 4 pm the experiment was exposed to 270 users with 13 of them taking action
(Click "Pay Now" button) with a conversion rate 4.8%.
The experiment was left on to which the results improved. The experiment ended up getting shown to 5,945 users with 323 taking action,
increasing the conversion rate to 5.4%.
Action EOM Payment Reminder Pay Now To Payment Experiment (All users)
As part of release 1 of the experiment, users are directed to the payments dashboard when the "Pay Now" button is clicked. Of the 323 users
who clicked the "Pay Now" button 6 users went on to make a payment, with a conversion of 1.9%.
Although these metrics were not part of the success criteria for the experiment there is an opportunity to improve overall conversions by
removing barriers presented on the Payments Dashboard.
EOM Payment Reminder Cancel Forever & Just For This Month Results
Of the 5,945 users who had seen the payment reminder notification experiment 124 (2.1%) users clicked the cancel/mute button on the
experiment, of this 36 opted to disable the notification forever, with a rate of 0.6%.
Customers also had the option to mute the notification for the month with the interest of being exposed to the notification again the following
month. 28 user muted the notification for the month a rate of 0.5%.
The low number of users opting to disable the notification forever suggests that it is helpful to them even though they may not action the
payment immediately.
Correlation between customers seeing the notification reminder and payments being processed
Insights
When users are exposed to the End of month statement notification we see an increase in customers making payments. This suggests that the
primary goal for the EOM statement notification is to increase awareness that there are statements available for payment and the 2nd goal is for
customers to be able to pay statements from the notification.
Of the 5,945 users who had seen the payment reminder notification experiment 625 went on to submit a payment with a conversion rate 10.5%.
Correlation between customers seeing the notification reminder and payments being processed
Insights
We know for a fact that accounts have multiple admin users and not all of them are responsible for paying the account. When we remove these
users and just leave users who have made payments in the past 60 days we see some interesting results.
1,741 were exposed to the payment reminder experiment.
53 clicked the "Pay Now" button creating a conversion rate of 3.0%.
625 who had seen the notification went on to make a payment, with a conversion rate of 35.6%.
Takeaways
Run the End of Month experiment again and target a small controlled cohort of users (100-200) who are responsible for making payments,
who pay statements rather than invoices, and who regularly pay End of Month (EOM) as a payment option.
Run the experiment on the app for IOS or Android to determine if the user's behavior is different from web users.
Run the experiment during weeks 1, 2 or 3 of the month as this aligns more closely with the messaging and when this notification would be
shown to customers.
Change the destination of the "Pay Now" button from Payments Dashboard to Quickpay with the statement value populated for payment to
attempt to improve end-to-end conversions.
Change the primary success metric from the "Pay Now" button to Payments made.
Despite the notification being seen by other admins who might not be responsible for making payments they appeared not bothered by the
message as seen by the low number of notification cancellations forever.
See if we can find out other payment methods. We aren’t currently tracking payments made by BPAY,
in branch, Direct deposit. Our data doesn’t currently add this information. We also know from interviews,
we are often the largest expense for a business and they often need to mentally prepare and plan to make a payment.
Add more events to Amplitude.
Timing
We know many of our small businesses like to pay at the last minute, keep the money in their account for as long as possible.
They reconcile during the month and pay within the last 5 days. If we surface the yellow reminder, will this be of more benefit for the customer.
Many of our customers like to be organised and don’t like to go past due on their account so a red alert would most likely prompt quick payment.
Maze test
Qualitative and Quantitative
In conjunction with notifying customers on the max dashboard we wanted to make it clearer on the Payments dashboard as well.
While doing the OKR work we continued with improving the payment experience and spoke to similar internal stakeholders. Both pieces were timely and complimentry.
There as feedback about customers paying the wrong “bucket” for example they would pay “recent” instead of the oldest statement first.
Research hypothesis
We believe that customers are unsure about which statement is due for payment first
so if we use clearer language and prioritise actions on the screen for the most urgent statements
we will see see an increase in customers paying the right statement with less calls to finance asking for an extension.
Success metrics
Our goal for the research, at 70% of customers successfully complete scenarios about making
payments and at least 70% of customers understand the information on the dashboard.
Find out if the content useful.
Questions
Can you locate on the screen, your available credit?
How would you go about paying the balance of your whole Reece max account?
How would you go about paying your overdue balance?
How would you go about paying your oldest invoice?
How would you go about paying a recent invoice?
How would you go about downloading your June statement?
Where would you go to see recent transactions that have been made against your account?
Research method
A dashboard design mini prototype for both desktop and mobile. We will conduct several
interviews with customers to get a deeper understanding of their thoughts around the dashboard
design and also send the Maze test to people who fit the AP criteria (ie, responsible for paying their
accounts and are a credit customer. Several questions/scenarios will be given and customers asked how they
would go about finding information or next steps for completing a task.
How would you go about paying the balance of your whole Reece max account?
Average success 86.7%. Average time on task 13-17 seconds.
People were able to complete the task but there was a high misclick rate.
Average SEQ score 4.6 (16 responses)
Over 70% of people found this task easy to complete
Why did you give it that rating?
Open question
How would you go about paying your overdue balance?
Over 90% of testers completed the mission. The average time on task was 5-9 seconds
Average success 93-100%.
People were able to complete the task and there was a low misclick rate
Average SEQ score 4.6 (16 responses)
Over 70% of people found this task easy to complete
Why did you give it that rating?
Open question