Epistimio / orion

Asynchronous Distributed Hyperparameter Optimization.
https://orion.readthedocs.io
Other
283 stars 52 forks source link

Deprecation Message proposes incorrect storage configuration #1019

Open lebrice opened 2 years ago

lebrice commented 2 years ago

Using the configuration for the database from the documentation:

https://orion.readthedocs.io/en/stable/user/config.html#full-example-of-global-configuration

database:
  host: /network/scratch/n/normandf/mup/runs/gpt2_256/orion_db.pkl
  type: pickleddb

This warning gets printed:

2022-11-03 12:26:14,259::WARNING::orion.core.io.resolve_config::(DEPRECATED) Option `database` and will be removed in v0.3. Use instead the option
storage:
  database:
    host: /network/scratch/n/normandf/mup/runs/gpt2_256/orion_db.pkl
type: pickleddb

However, this proposed format doesn't work:

2022-11-03 12:28:30,399::WARNING::orion.core.io.resolve_config::(DEPRECATED) Option `type` and will be removed in v0.3. Use instead the option
storage:
  type:
    pickleddb
...

Traceback (most recent call last):
  File "/network/scratch/n/normandf/conda/mup/bin/orion", line 8, in <module>
    sys.exit(main())
  File "/network/scratch/n/normandf/conda/mup/lib/python3.9/site-packages/orion/core/cli/__init__.py", line 36, in main
    return orion_parser.execute(argv)
  File "/network/scratch/n/normandf/conda/mup/lib/python3.9/site-packages/orion/core/cli/base.py", line 110, in execute
    returncode = function(args)
  File "/network/scratch/n/normandf/conda/mup/lib/python3.9/site-packages/orion/core/cli/hunt.py", line 192, in main
    experiment = experiment_builder.build_from_args(args)
  File "/network/scratch/n/normandf/conda/mup/lib/python3.9/site-packages/orion/core/io/experiment_builder.py", line 364, in build_from_args
    builder = ExperimentBuilder(cmd_config["storage"], debug=cmd_config.get("debug"))
  File "/network/scratch/n/normandf/conda/mup/lib/python3.9/site-packages/orion/core/io/experiment_builder.py", line 442, in __init__
    self.storage = setup_storage(storage, debug=debug)
  File "/network/scratch/n/normandf/conda/mup/lib/python3.9/site-packages/orion/storage/base.py", line 600, in setup_storage
    return storage_factory.create(of_type=storage_type, **storage)
  File "/network/scratch/n/normandf/conda/mup/lib/python3.9/site-packages/orion/core/utils/__init__.py", line 128, in create
    constructor = self.get_class(of_type)
  File "/network/scratch/n/normandf/conda/mup/lib/python3.9/site-packages/orion/core/utils/__init__.py", line 143, in get_class
    raise NotImplementedError(
NotImplementedError: Could not find implementation of BaseStorageProtocol, type = 'pickleddb'
Currently, there is an implementation for types:
['legacy', 'track']
bouthilx commented 2 years ago

Oh, the type is not indented properly. If it was at the level of database it would work. Thanks for reporting!

Delaunay commented 1 year ago

Problem seems to be on that line https://github.com/Epistimio/orion/blob/develop/src/orion/core/io/resolve_config.py#L237

yaml.dump(backward_value, indent=6), probably does not handle multi line that well