Feature: Linearize log distributions in TPE for proper processing of the hyperparams in the SVD

[guillaume-chevalier]

Is your feature request related to a problem? Please describe. Log-shaped distributions are expected to perform poorly in the Orthogonal (SVD) TPE and should be linearized.

Describe the solution you'd like Also, linearize dimensions before applying the SVD (e.g.: linearizing a log distribution to make it more uniform or more normal instead of skewed).

Ideas:

• Either use Mixins (with "pass") in distribution to easily tag which distribution is a log one to normalize them easily.
• Or perhaps try to featurize the hyperparameter differently and see what creates a more uniform distribution of it
• Or else even maybe create a .linearize() function on the HyperparameterSpace object to recursively have each distribution linearize itself (if needed) with a new abstract function Distribution.linearize() that each distribution would inherit from. There would also be the inverse .delinearize function.

I think the first idea is the simplest.

Describe alternatives you've considered @Eric2Hamel tried using Log distributions for the sampled good and bad points. Although this is OK for the regular TPE, passing this to the SVD will likely make the SVD fail or underperform.

Additional context

464

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs in the next 180 days. Thank you for your contributions.