search

JuliaAI / MLJModels.jl

Home of the MLJ model registry and tools for model queries and mode code loading
MIT License
80 stars 27 forks source link

Non-Dummy variable codings in `ContinuousEncoder` #534

Open ParadaCarleton opened 11 months ago

ParadaCarleton commented 11 months ago

Right now, MLJModels.jl has ContinuousEncoder, which automatically transforms into one of two codings:

  1. Dummy variable coding (one-hot with last category dropped)
  2. Redundant variable coding (one-hot)

However, these aren't always the most useful codings for effective regularization, and there are many others in common use. For example, the most common way to encode ordinal variables is with sequential difference encoding; with this encoding, regularization pulls adjacent categories closer together, which improves model performance relative to either treating ordered variables as categorical (discarding ordering information) or treating them as continuous (using an equal-distance assumption that is often incorrect). Similarly, effect coding allows you to regularize categories towards the grand mean (rather than regularize every category towards 0, or regularize all categories towards one other category).

ablaom commented 11 months ago

Sounds like a good idea to me. PR welcome :wink:

ablaom commented 7 months ago

Summer project: https://julialang.org/jsoc/gsoc/MLJ/#categorical_variable_encoding