ZERO Fraud
Analytic Dashboard
Project Specs
My Role: Web Dashboard Design
Timeline: Fall 2018. 2 Months
Scope: UX design and prototype given design brief
Context
This is a project of cooperation between real fraud management company and class to solve a real-world problem.
ZERO Fraud (substitute name) is a decision analytics company that has worked with some of the world’s largest credit card companies to screen transactions and highlight suspicious activity. This independent school project is about designing a tool for anti-fraud managers to efficiently monitor case queues, analysts, and model performance, and take actions when necessary.
Goal
The focus of this design project would be improving the experience of how managers view their current anti-fraud strategy -- comprised of people and models, and take actions to improve that strategy if needed.
Process
Persona
Team Leader/Anti-Fraud Manager : Janine
Typical Workday
She needs to monitor the productivity of queues, making sure that all queues and analysts' performance are up to standard.
A Perfect Day
Instead of having to react to operational challenges in a knee- jerk fashion, she is able to proactively run scenarios that consider likely challenges, so that she has a suite of countermeasures ready to quickly deal with operational challenges.
Pain Points
- Need to make high-paced decision making
- High information load
- Hard to drill down the data she needs
- Reports don’t provide clear visual communication
- Hard to make a decision through disparate systems
Problem Analysis
As an anti-fraud manager, Janine's top priority is to keep the queue size stable so that cases don't get piled up.
- The queue size is determined by 2 factors
1) How fast are new cases coming in?
2) How fast are cases being closed (average worker speed)?
The two factors can be combined into the formula below
Rate of Change of Queue Size =
New Cases Flow-in Rate - (Number of Workers x Average Worker Speed)
- Why is the queue piling up?
Scenario 1) There's an influx in new cases flow-in rate
Solution: Add more analysts to queue, ideally from queues that are not so busy or have excess analysts.
Scenario 2) Cases are becoming more difficult to solve, or that the analysts are underperforming
Solution: Review average worker speed, go check on the slow workers
As a result, I chose a design that highlights different aspects of the queue behavior, with an AI that can proactively identify queue performance problems, leveraging the formula above.
Proposed Solutions
Efficient Zone
If the queue size exceed the efficient zone, the queue is piling up.
If it's below the efficient zone, there are too many analysts in queue, whom can be assigned to other queues in need.
Monitor and AI Prediction
Show both the flow-in rate and analysts speed of the queue, and highlight the alert if applicable.
AI can predict if the queue will exit efficient zone before it happens, so Janine can respond proactively.
AI Action Board
Besides showing the prediction, AI will calculate all the possible solutions and create a list of suggested mitigation.
Efficient Zone, Monitor and AI Prediction
The alert shows the predictive queue size will exceed efficient zone in an hour.
The AI Action board
Instead of letting Janine to digest and access the problems, the AI can analyze the situations and provide suggested solutions.
Measure and Dimension
Identifying measures (raw data) and dimensions (normalized data) can help me decide what is the more important information to display, building hierarchical content within the dashboard.
Here is one example showing measures and dimensions for workers:
Analysts (Workers)
Measure
- How long they work a queue and which queue?
- How many cases had a status? What that status was ? How many cases per hour/day?
- How fast they are working cases ?
- What are their specialties?
Dimension
- Usage of time in percentage
- Compare to different date range
- Compare individuals to all
- Compare within individuals
- Ranking within queues
For more analysis on design grammar, priority matrix, measures and dimensions for queue, case, and model, please click here.
Final UI
Queue Page
- Select a queue
- Examine flow-in rate and worker speed
- Check prediction and alert
- AI action boards
- Analyst performance over time in queue
Model Performance
Because model adds new cases to queues, it's important that Janice monitors the False Positive Rate (FPR) of the models. She can then drill in to cases denoted to FPR and view the details of the case.
Case Detail
This page contains the case summary, timeline and all the analysts who have worked on the case. Janine can comment at any steps, and see the analyst profile if she needed.
Analyst Profile
Janine can drill down to see the particular analyst profile whenever the name shows up. The profile page includes the analyst's performance summary, the time investment in each queue, and the comparison to other analysts.
Overview - Dashboard Home Page
This page allows Janine to see the whole picture, survey the performance of each important part of the system.
The overview consists of:
- Queue performance (size growing rate)
- Team performance (team size, the goal speed needed to clear the queue, and the problem analysis that provides details)
- Model performance (false positive rate)
View clickable prototype here.
Conclusion
In summary, the dashboard achieves the following:
- Provide a coherent experience for fraud managers to see the big picture.
- The prediction and alert features allow fraud managers to react to problems proactively.
- AI suggested action boards enhance decision making, reduce calculation effort for fraud managers.
- Display the crucial data that governs the current performance, use data visualization to enhance understanding.