Support Scorers

[schlichtanders]

Hi, just started to look into ScikitLearn.jl and am very surprised finding that it has almost no support for scorers in crossvalidation

function get_scorer(scoring::Symbol)
    if haskey(SCORERS, scoring)
        return SCORERS[scoring]
    else
        throw(ArgumentError("$scoring is not a valid scoring value. Valid options are $(sorted(keys(SCORERS)))"))
    end
end

const SCORERS = Dict(
                     ## r2=r2_scorer,
                     ## median_absolute_error=median_absolute_error_scorer,
                     ## mean_absolute_error=mean_absolute_error_scorer,
                     :mean_squared_error=>mean_squared_error_scorer,
                     ## accuracy=accuracy_scorer, roc_auc=roc_auc_scorer,
                     ## average_precision=average_precision_scorer,
                     ## log_loss=log_loss_scorer,
                     ## adjusted_rand_score=adjusted_rand_scorer)
                     )

also, the get_scorer method does not throw the ArgumentError, but fails because sorted is not defined.

Is this missing support for Scorers intended? What is the general commitment of ScikitLearn.jl in this respect?

[cstjean]

Is this missing support for Scorers intended? What is the general commitment of ScikitLearn.jl in this respect?

Commitment? None, really. We'd love to support more, and would welcome a PR, but ScikitLearn isn't very actively developed. As far as I can tell it works reasonably well for its users, but it's more of a gateway package into the ecosystem.

If you're comfortable with Julia already, and don't have a strong sklearn background, I'd recommend you check out MLJ. It's a lot more active.

[schlichtanders]

Thank you very much for the clear answer. I myself don't have time either to implement the scorer features.