search

evidentlyai / evidently

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
https://www.evidentlyai.com/evidently-oss
Apache License 2.0
5.37k stars 596 forks source link

Target drift report with real label only #1306

Open xuyunlong123 opened 1 month ago

xuyunlong123 commented 1 month ago

how can i use the target drift detect for feature, i don't find in the doc.

elenasamuylova commented 1 month ago

Hi @xuyunlong123, could you clarify what you try to achieve?

If you only have a single column (real label) and want to see the shift in its distribution, you can simply create a Report with a single ColumnDriftMetric for this column.

If you want to detect drift for a set of features, you use DataDriftPreset.

The TargetDriftPreset is useful when you have both prediction/label column and feature columns and you want to look at data drift as well as on correlation changes / visually explore the dependency between target and features.

xuyunlong123 commented 1 month ago

Hi @xuyunlong123, could you clarify what you try to achieve?

If you only have a single column (real label) and want to see the shift in its distribution, you can simply create a Report with a single ColumnDriftMetric for this column.

If you want to detect drift for a set of features, you use DataDriftPreset.

The TargetDriftPreset is useful when you have both prediction/label column and feature columns and you want to look at data drift as well as on correlation changes / visually explore the dependency between target and features.

emm，i see. the situation is that i creat many creaditscore models for loan application,but they have the same target label and different features, so i want to use one detector which can test the drift between features and target.That confuses me.Thanks for your reply.

xuyunlong123 commented 1 month ago

Hi @xuyunlong123, could you clarify what you try to achieve?

If you only have a single column (real label) and want to see the shift in its distribution, you can simply create a Report with a single ColumnDriftMetric for this column.

If you want to detect drift for a set of features, you use DataDriftPreset.

The TargetDriftPreset is useful when you have both prediction/label column and feature columns and you want to look at data drift as well as on correlation changes / visually explore the dependency between target and features.

I have another question if i only have label and feature columns, the TargetDriftPreset could work?

elenasamuylova commented 1 month ago

Hi @xuyunlong123, yes - you just need to label your label column as target in column mapping https://docs.evidentlyai.com/user-guide/input-data/column-mapping#target-and-prediction