Exploration - Getting Started with Evidently

[manisnesan]

Expected Outcomes

• Follow the Getting Started and Run through the tutorial step by step guide for iris datasets

[manisnesan]

Notes

Functionality

• Model Health
• Data Drift
• Target Drift

Requirements Reference Data - Most likely the test data set Current Production Data

What should be used as reference & current???

In practice, you can use it in different combinations:

• Training vs Test : To compare the model performance on a hold-out Test to the Training. Pass the training data as "Reference", and test data as "Current".
• Production vs Training : To compare the Production model performance to the Training period. Pass the training data as "Reference", and production data as "Current".
• Current perfromance vs Past: To compare the Current production performance to an Earlier period. For example, to compare the last week to the previous week or month. Pass the earlier data as "Reference", and newer data as "Current".
• Compare any two models or datasets: For example, to estimate the historical drift for different windows in your training data or to compare how two models perform in the test. Pass the first dataset as "Reference", and the second as "Current".

For Data Drift report, include the input features only. For Target Drift reports, include the column with Target and/or Prediction. For Model Performance reports, include the columns with Target and Prediction.

Generates a dashboard with HTML report or json profiles