search

dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
8.94k stars 1.86k forks source link

Predicted values are not at original scale after applying preFeaturizer #6801

Open superichmann opened 11 months ago

superichmann commented 11 months ago

TLDR: How can I do inverse transform using TransformerChain?

When applying a preFeaturizer on the target column var logTransformer = mlContext.Transforms.NormalizeMinMax("Label",fixZero:true); within a regression experiment, the predicted values IDataView preds = result.Model.Transform(predictMe); are in the wrong scale.

Original values: 10 22 33 12 6 6 5

Predictions without preFeaturizer: 8.593365 19.133606 19.094164 11.576735 11.040246 10.156776 9.546574

Predictions with preFeaturizer: 0.06661523 0.14832252 0.14801677 0.08974213 0.08558331 0.0787347 0.07400444

as you can see, the predictions with the preFeaturizer applied are in a totally different scale.

How can I actually get the predicted values in the scale of the original target column and not in the scale of the applied transformer?

superichmann commented 8 months ago

@LittleLittleCloud Can you please take a look?

LittleLittleCloud commented 8 months ago

In your case, I would just remove NormalizeMinMax on label column, as NormalizeMinMax is a linear transformer I wonder if applying it on the label would make a difference on the training result.

If you still want to keep the transformer, you would need to scale the predicting label back manually

// y_min, y_max is the minimum and maxmium label value from your training dataset
y_normalized = (y - y_min) / (y_max - y_min)
y = y_normalized * (y_max - y_min) + y_min