search

mlr-org / mlr3pipelines

Dataflow Programming for Machine Learning in R
https://mlr3pipelines.mlr-org.com/
GNU Lesser General Public License v3.0
137 stars 25 forks source link

PipeOpThreshold has wrong predict type #712

Closed sebffischer closed 2 weeks ago

sebffischer commented 1 year ago

PipeOpThreshold claims to have predict_type "response" but actually outputs probabilities. Also, why is it not possible to change the predict type of the PipeOp?

mb706 commented 1 year ago

The problem seems to be that PipeOpLearnerCV does not have the right predict_type. PipeOpTuneThreshold does not claim to have any predict type, but requires the incoming task to have probability predictions. The graph infers the predict type from the last pipeop in the (sequential) graph that announces its predict type. PO TuneThreshold does not announce any type, but PipeOpLearnerCV does -- just the wrong one.

mb706 commented 1 year ago

Never mind, PipeOpLearnerCV also does not have any predict_type, since it is not a learner. "response" is just the default for a GraphLearner.

mb706 commented 1 year ago

@mllg the "interesting" part of PipeOpTuneThreshold output is the response prediction. But since the probability prediction is present, that is returned as well. Should a GraphLearner that does threshold tuning, but where probabilities are also returned, have predict_type "prob" or "response"? I would give it "prob", but I am wondering if there is any part of mlr3 that prioritizes probability predictions unless a Learner has predict type "response"? (Or that prioritizes probabilities full stop, forcing PipeOpTuneThreshold to drop the probability predictions altogether so it has an effect?)