search

denisyarats / drq

DrQ: Data regularized Q
https://sites.google.com/view/data-regularized-q
MIT License
407 stars 52 forks source link

macOS Catalina can't run due to Hydra #11

Closed avandekleut closed 4 years ago

avandekleut commented 4 years ago

After following the installation instructions, I run into a problem with Hydra:

HYDRA_FULL_ERROR=1 python train.py env=cartpole_swingup                                                
Traceback (most recent call last):
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/config_loader_impl.py", line 153, in load_configuration
    from_shell=from_shell,
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/config_loader_impl.py", line 256, in _load_configuration
    run_mode=run_mode,
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/config_loader_impl.py", line 796, in _merge_defaults_into_config
    hydra_cfg = merge_defaults_list_into_config(hydra_cfg, system_list)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/config_loader_impl.py", line 764, in merge_defaults_list_into_config
    merged_cfg.merge_with(job_cfg)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/basecontainer.py", line 325, in merge_with
    self._format_and_raise(key=None, value=None, cause=e)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/base.py", line 101, in _format_and_raise
    type_override=type_override,
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/_utils.py", line 610, in format_and_raise
    _raise(ex, cause)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/_utils.py", line 591, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/basecontainer.py", line 323, in merge_with
    self._merge_with(*others)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/basecontainer.py", line 341, in _merge_with
    BaseContainer._map_merge(self, other)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/basecontainer.py", line 288, in _map_merge
    dest_node._merge_with(src_value)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/basecontainer.py", line 341, in _merge_with
    BaseContainer._map_merge(self, other)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/basecontainer.py", line 308, in _map_merge
    dest[key] = src._get_node(key)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/dictconfig.py", line 251, in __setitem__
    key=key, value=value, type_override=ConfigKeyError, cause=e
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/base.py", line 101, in _format_and_raise
    type_override=type_override,
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/_utils.py", line 610, in format_and_raise
    _raise(ex, cause)
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/omegaconf/_utils.py", line 591, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
omegaconf.errors.ConfigKeyError: Key 'name' not in 'HydraConf'
    full_key: hydra.name
    reference_type=Optional[HydraConf]
    object_type=HydraConf

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "train.py", line 178, in <module>
    main()
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/main.py", line 37, in decorated_main
    strict=strict,
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/utils.py", line 356, in _run_hydra
    lambda: hydra.run(
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/utils.py", line 210, in run_and_report
    raise ex
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/utils.py", line 207, in run_and_report
    return func()
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/utils.py", line 359, in <lambda>
    overrides=args.overrides,
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/hydra.py", line 104, in run
    run_mode=RunMode.RUN,
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/hydra.py", line 512, in compose_config
    from_shell=from_shell,
  File "/usr/local/Caskroom/miniconda/base/envs/drq/lib/python3.7/site-packages/hydra/_internal/config_loader_impl.py", line 156, in load_configuration
    raise ConfigCompositionException() from e
hydra.errors.ConfigCompositionException

I have looked through the stack trace and am not sufficiently familiar with Hydra or OmniConf to decipher what is actually causing the issue. Maybe we have different versions for the packages that got installed from the conda_env.yml file?

Yingdong-Hu commented 4 years ago

I have the same problem.

denisyarats commented 4 years ago

It looks like you are using a newer hydra version. I would guess it is 1.*? The version that I was used for this project was '0.11.3', try to install this version? If this doesn't work I can modify the code to work with the newer version.

denisyarats commented 4 years ago

It looks like we just need to replace this line: https://github.com/denisyarats/drq/blob/master/train.py#L170 with this

@hydra.main(config_name="config")

Could you try to do that and let me know if that works?

Yingdong-Hu commented 4 years ago

I replace config_path=config.yaml with config_name=config, but there is another problem. The doc say 'Hydra 1.0.0 is deprecating ObjectConf and the corresponding config structure to a simpler one without the params node'. It seems that more code should be modified.

denisyarats commented 4 years ago

Ok, I will investigate and get back to you.

On Fri, Sep 11, 2020 at 8:45 AM Alex notifications@github.com wrote:

I replace config_path=config.yaml with config_name=config, but there is another problem. The doc say 'Hydra 1.0.0 is deprecating ObjectConf and the corresponding config structure to a simpler one without the params node'. It seems that more code should be modified.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/denisyarats/drq/issues/11#issuecomment-691138379, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZYO4HNZ3OMGRKZ5PUOLODSFIZXTANCNFSM4RGOA5AQ .

avandekleut commented 4 years ago

I was able to get it to run by installing Hydra 0.11 directly from the GitHub repo's branch, like so:

pip install git+git://github.com/facebookresearch/hydra@0.11_branch

I also had to change "cuda" to "cpu" in config since I was testing on my MacBook.

denisyarats commented 4 years ago

Awesome, could you please push an update to conda_env.yaml to make sure it installs 0.11 version and create a PR?

On Fri, Sep 11, 2020 at 5:28 PM Alexander Van de Kleut < notifications@github.com> wrote:

I was able to get it to run by installing Hydra 0.11 directly from the GitHub repo's branch, like so:

pip install git+git://github.com/facebookresearch/hydra@0.11_branch

I also had to change "cuda" to "cpu" in config since I was testing on my MacBook.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/denisyarats/drq/issues/11#issuecomment-691353090, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZYO4F4PCDL3EZDHZUG6ZDSFKXCZANCNFSM4RGOA5AQ .

avandekleut commented 4 years ago

Closing issue due to merged pull request.

avandekleut commented 4 years ago

Due you want me to make a pull request for a version of the code with hydra dependencies removed?

denisyarats commented 4 years ago

Sure, that would be nice!

On Mon, Sep 21, 2020 at 3:50 PM Alexander Van de Kleut < notifications@github.com> wrote:

Due you want me to make a pull request for a version of the code with hydra dependencies removed?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/denisyarats/drq/issues/11#issuecomment-696335028, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZYO4H6U73J77TTZVXHBITSG6VAHANCNFSM4RGOA5AQ .