Customize hyper-parameters for pipelines

[ZhuofanXie]

• Orion version: v0.1.5
• Python version: 3.7
• Operating System:

Description

In the tutorial notebook, it is recommended that "For experimentation purposes, you can reduce the number of epochs in the tadgan.json file such that you reduce the number of training iterations."

This operation is risky in the sense that tadgan.json is in some sense considered the default pipeline file. If the users forget to change it back (which is very likely if they are only doing an experimental run in their first trial), it might lead to undertraining in later runs. Also, this operation is inconvenient, especially when the users want to try a couple hyper-parameter(s) to find out the best fit for their data sets.

One solution might be to remind the users in the tutorial notebook that they can first load the json files as dictionaries, change the hyper-parameter(s) inside and then pass it into the Orion factory method.

Alternatively, like many other classes (e.g. sklearn models), we can give the users the option to pass in additional keyword arguments (or as dictionaries) to overwrite the ones in the json files. That is, creating objects in these ways:

orion = Orion(
    pipeline='tadgan.json', epochs=10
)

or

orion = Orion(
    pipeline='tadgan.json', kwargs={epochs : 10}
)

[sarahmish]

Thanks @ZhuofanXie for the suggestion!

Some ideas on how to proceed with this feature:

• In pipeline jsons file, specify the global name for the hyperparameter, for example epochs and which primitive it maps to. A new section in the json file is proposed where we map keyword arguments to primitives and their respective hyperparameters
• In Orion API, we can search for any primitive that has the specified hyperparameter name, and change that for all hyperparameters. In some cases, this will lead to an undesired behavior.

"hyperparameters": {
    "mapping": [
        "epochs": "keras.Sequential. LSTMTimeSeriesRegressor#1.epochs"
    ]
}