search

paris-saclay-cds / ramp-workflow

Toolkit for building predictive workflows on top of pydata (pandas, scikit-learn, pytorch, keras, etc.).
https://paris-saclay-cds.github.io/ramp-docs/
BSD 3-Clause "New" or "Revised" License
68 stars 42 forks source link

[RFC] Score refactoring #169

Closed glemaitre closed 5 years ago

glemaitre commented 5 years ago

Scores are defined as classes. IMO, this is an over-complicated design.

I think that those classes could be replaced by functions. If we want to keep the class first citizen then we could only do a wrapper to create classes from functions. What I am unsure is the impact on the backend to shift paradigm.

agramfort commented 5 years ago

currently higher is not always better for the scores and it allows to have multiple scores

but I agree that the make_scorer from sklearn would allow not to reinvent the wheel...

glemaitre commented 5 years ago

currently higher is not always better for the scores and it allows to have multiple scores

Make it negative then :)

agramfort commented 5 years ago

I know :)

kegl commented 5 years ago

I would say don't touch something that's working fine. We have plenty features to implement. I wrapped sklearn's scores on purpose into classes to be able to have paramterizable scores and getting rid of that IMHO bad design negative RMSE (for example).

kegl commented 5 years ago

I'd close this issue for now.