Back to Sift Support Plans Submit a request

How can we help?

Ask a question or search our knowledge base:

Workflow Route Metrics

Overview

Workflow Route Metrics help you quickly understand the performance of your workflow as well as the performance of individual routes. This feature is available for any running Workflow, and you can see data starting from any published version within the past 30 days. 

accuracy_metrics.png

Key features

  1. Workflow-level Summary Metrics: See a rollup of performance metrics for your entire Workflow, including total number of runs, unique entities that triggered the Workflow, overturned block decisions, and an estimated missed fraud rate. 

  2. Route-level Visualizations: Explore the performance of individual Workflow routes in volume-based, score-based, and decision-based graphs and charts. 

Note: Metrics calculations have a roughly 15 minute delay. We subsequently cache the data for a few minutes in your browser, so you may not immediately see metrics change if you examine them multiple times within a short period of time. 

Summary Metrics 

After selecting any running Workflow, you'll be able to see summary of performance metrics for the entire Workflow. If the Workflow applies decisions to users, then we'll show the unique number of users (based on the User ID) that went through the Workflow. If the Workflow applies decisions to another entity, like orders, content, or sessions, then we'll show  the unique number of users and the unique number of entities that went through the Workflow. 

workflow_level_summary_metrics.png

This is also where you can toggle the time range for all of the Route Metrics shown on this page. You can select the Last 30 Days to see the performance metrics for all edits made within that time period. You can also choose to review metrics starting from any published version of the Workflow within the last 30 days. 

date_picker.png

You will also be able to see a similar set of summary metrics for each route in your Workflow. 

route_level_summary_metrics.png

Volume Graph 

For routes with a non-zero number of runs, you can click on the chevron (“v”) in the bottom right corner to reveal the a variety of graphs and charts. The first graph shown is the Volume Graph. 

The Volume Graph shows you a visualization of the volume of runs for the given route over time. If the time period of the chart is less than 1 day, you will also be able to toggle between daily or hourly increments. You can toggle components of the graph on/off by using the legend in the top right of the graph. 

Hover over any bar in the graph to see the exact count and percentage of total Workflow runs for that time period. If there are multiple edits made to a Workflow over the given time period, these edits are indicated with vertical dotted lines at the time that each version was published. 

volume_no_metrics_no_download.png

Use the Volume Graph to answer questions like:

  • How many users/entities is the route is catching?

  • Is my review volume trending up or down?

  • What percentage of my signups did I block on a given day?

  • How has the performance of this route changed since my last edit?  

Latest Decisions Table 

Click “Latest Decisions“ to see a table of the latest Decision for every workflow run that happened in the time period. Click to expand a row and see a breakdown of Decisions for that category:

latest_decisions_no_download.png

Use the Latest Decisions Table to answer questions like:

  • What are the different decisions that are ultimately applied to the entities that go through this route? 

  • Does my block route have a surprisingly large percentage of accept decisions applied? Should I consider adjusting the criteria to catch fewer good users? 

Accuracy Graph 

The Accuracy Graph shows you a visualization of the number of entities that were caught by the route graphed by the Sift Score and color-coded by the latest decision applied to that entity. If the time period of the chart is less than 1 day, you will also be able to toggle between daily or hourly increments. You can toggle components of the graph on/off by using the legend in the top right of the graph - this can be helpful if the volume of a particular decision category is too large. 

Hover over any bar in the graph to see the exact count of each decision category at given score threshold and a sum of counts above or below the given threshold. You can toggle whether the hover panel shows counts above or below the given score threshold by using the dropdown in the top right of the graph. 

accuracy_no_metrics_3.png

 Use the Accuracy Graph to answer questions like:

  • What is the score distribution for the users/entities caught by my route? 

  • Is there a large number of accept decisions being applied to users in this queue? Should I narrow the thresholds for this route? 

  • Is my block route applying decisions that are maintained, or are the auto-block decisions being overturned? 

  • How does my score threshold correlate with chargebacks? 

CSV Download 

You can export the underlying Route Metrics data to a CSV up to the last 10k runs. This export will include the user ID, entity ID, latest decision type, latest decision source, score, and timestamp. Use this export to validate the metrics you see for your Workflows, or to find individual users/entities to explore in the Sift Console. 

Note: This feature is currently not supported for Workflows that use multiple Abuse Types.

csv_download_button.png

Descriptions and Tooltips

If you are unsure of the meaning of any metric, simply click on it to read a description of the underlying data and calculation. 

18dbf9f7-8d92-4701-a2e5-571563808874.png

There are also a variety of tooltips to help you interact with the metrics and visualizations. 

a3694aaa-8731-44a3-9cac-be6b2ad62d6e.png

If you have any feedback or any more questions, please don’t hesitate to reach out to us at support@sift.com.

Have more questions? Submit a request