Use MLJTuning.jl for ranges

[MilesCranmer]

It would be really nice if MLJTuning.jl's ranges worked out of the box: https://alan-turing-institute.github.io/MLJ.jl/dev/tuning_models/. Right now it looks like one needs to manually translate the ranges from, e.g.,

range(model, :x, lower=-5, upper=5)

to

Dict(:x => HP.Uniform(:x, -5., 5.))

But since all of the other tuning packages use the first API it would be better if one could just switch between the other tuning packages and this one!

Cheers, Miles

[kainkad]

hi @MilesCranmer thanks for raising the issue. I looked at what it takes to implement the MLJTuning ranges API but there seem to be several limitations for implementing this method in MLJTreeParzen:

• The range API only allows for a single one-dimensional range or vector of one-dimensioinal ranges can be specified (e.g.: range(model, :hyper1, lower=1, origin=2, unit=1), or [(range(model, :hyper1, lower=1, upper=10), 15), range(model, :hyper2, lower=2, upper=4), range(model, :hyper3, values=[:ball, :tree])])
• There is a limited number of supported distributions that can be passed to the range API (e.g. based on how it’s handled in the RandomSearch: https://juliaai.github.io/MLJ.jl/dev/tuning_models/#MLJTuning.RandomSearch)
• Related to the one-dimensional range of a vector of ranges the key limitation (correct me if I'm wrong) is that range() function in MLJTuning doesn't support nested choices like in the space example below:

  space = Dict(
  :is_unbalance => HP.Choice(
      :is_unbalance,
      [
          Dict(:is_unbalance => true),
          Dict(
              :is_unbalance => false,
              :scale_pos_weight => HP.QuantUniform(:scale_pos_weight, 1., 10., 1.)
          )
      ]
  )
  )

  where the nested HP.Choice allows to allocate different true/false behaviours for the :is_unbalance parameter. Otherwise, the logic of :scale_pos_weight would need to be handled differently and being only relevant when :is_unbalance is false in the model training.

Even if considering those limitations, we would probably be looking at some sort of error prone expression parsing involved to read those fields from the range() syntax and pass onto tree-parzen structure.

I'm not closing the issue as it's something to keep in view but given the above it doesn't seem feasible.