Tracking Metrics

Track Production Metrics

Description

  • Tracking Production Metrics enables users to plot trends in metrics related to model performance. These metrics can be model-specific metrics (ROC-AUC, accuracy, etc.) or business metrics.

  • Metrics tracked & scheduled as Monitors through Orbit's built-in scheduler show up as a plot titled with the metric name and the data series labeled as the monitor name. 

foundations.track_production_metrics(metric_name, metric_value_dictionary)
Arguments

  • metric_name (str): A name associated with the metric.

  • metric_value_dictionary (dict): A key-value pair of {str:int|float} where the key is the inference date formatted as "%Y-%m-%d %H:%M:%S" and the value is the evaluated metric value

Returns

  • None

Example:

import foundations
from datetime import datetime

metric_name = 'my_metric'
inference_date = str(datetime.strftime(datetime.now(), "%Y-%m-%d %H:%M:%S"))

calculated_metric_value = 0.7 #sample value
metric_values_dictionary = {inference_date: calculated_metric_value} 

foundations.track_production_metrics(metric_name, metric_values_dictionary)