dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
9.05k stars 1.89k forks source link

model.LastTransformer doesn't exist #4937

Closed philiplai closed 4 years ago

philiplai commented 4 years ago

I was trying to implement Permutation Feature Importance (PFI) for Binary Classification. But I was stuck on the following line of code. This method simply doesn't exist. // Extract the predictor. var linearPredictor = model.LastTransformer;

I was following the example on https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.permutationfeatureimportanceextensions.permutationfeatureimportance?view=ml-dotnet

Any idea?


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

najeeb-kazmi commented 4 years ago

@philiplai can you share the code where you load the data, define the pipeline, and fit the model?

LastTransformer is a property of TransformerChain. My best guess without looking at the code is that your model has only one transformer, and is not a TransformerChain. If that is the case, your model will be of type BinaryPredictionTransformer, which you can directly pass to PFI instead of the linearPredictor in the example.

artemiusgreat commented 4 years ago

A duplicate of this issue https://github.com/dotnet/machinelearning/issues/3976 The problem is that model.Load by default returns incorrect type that doesn't have property LastTransformer, so you have to cast it to the right data type explicitly.

antoniovs1029 commented 4 years ago

For the record, I wouldn't think that issue #3976 is a duplicate of this issue here. @artemiusgreat is right in that mlContext.Model.Load always return an ITransformer (link to docs). Since ITransformer doesn't have a .LastTransformer property, then it needs to be casted to a TransformerChain as shown in the following sample:

https://github.com/dotnet/machinelearning/blob/290da8222d4ebcf5c9c4fa134d27151d4ee69364/docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/BinaryClassification/PermutationFeatureImportanceLoadFromDisk.cs#L36-L41

The need for a cast to access the .LastTransformer is by design, and it isn't a bug.

On the other hand, issue #3976 was actually that attempting to do the cast above caused an exception, and so PRs #4262 and #4306 fixed this bug. The only reason #3976 is still open is because the issue specifically refers to using ML.NET on F#, and the fix made on those PRs haven't been tested on F#, only on C#.

Notice that an alternative cast that would also work would be:

var linearPredictor = (loadedmodel as TransformerChain<ITransformer>).LastTransformer as ISingleFeaturePredictionTransformer<object>;

As for the issue in here, I wouldn't know if the user who created the issue is trying to load a model (in which case, making the cast @artemiusgreat suggested might be enough), or if the user has a model with one transformer (as @najeeb-kazmi has suggested, in which case passing that predictor directly to PFI should be enough).

najeeb-kazmi commented 4 years ago

Closing this issue. Please feel free to reopen if something remains unanswered.