Trying to run it on AWS using dora.

makkarss929 commented 5 months ago

Traceback (most recent call last):
  File "/home/ubuntu/audiocraft/venv/bin/dora", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/dora/__main__.py", line 158, in main
    main = get_main(args.main_module, args.package)
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/dora/_utils.py", line 48, in get_main
    module = import_or_fatal(module_name)
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/dora/utils.py", line 77, in import_or_fatal
    return importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ubuntu/audiocraft/audiocraft/train.py", line 155, in <module>
    main.dora.dir = AudioCraftEnvironment.get_dora_dir()
  File "/home/ubuntu/audiocraft/audiocraft/environment.py", line 108, in get_dora_dir
    cluster_config = cls.instance()._get_cluster_config()
  File "/home/ubuntu/audiocraft/audiocraft/environment.py", line 81, in instance
    cls._instance = cls()
  File "/home/ubuntu/audiocraft/audiocraft/environment.py", line 68, in __init__
    cluster_config = self._get_cluster_config()
  File "/home/ubuntu/audiocraft/audiocraft/environment.py", line 76, in _get_cluster_config
    return self.config[self.cluster]
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/omegaconf/dictconfig.py", line 375, in __getitem__
    self._format_and_raise(key=key, value=None, cause=e)
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/omegaconf/base.py", line 231, in _format_and_raise
    format_and_raise(
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/omegaconf/_utils.py", line 899, in format_and_raise
    _raise(ex, cause)
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/omegaconf/_utils.py", line 797, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/omegaconf/dictconfig.py", line 369, in __getitem__
    return self._get_impl(key=key, default_value=_DEFAULT_MARKER_)
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/omegaconf/dictconfig.py", line 442, in _get_impl
    node = self._get_child(
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/omegaconf/basecontainer.py", line 73, in _get_child
    child = self._get_node(
  File "/home/ubuntu/audiocraft/venv/lib/python3.10/site-packages/omegaconf/dictconfig.py", line 480, in _get_node
    raise ConfigKeyError(f"Missing key {key!s}")
omegaconf.errors.ConfigKeyError: Missing key aws
    full_key: aws

1Isra2 commented 4 months ago

Hey @makkarss929 , any luck sorting out this issue? I'm getting exactly the same error when executing the "dora run" command on a Docker Container running as a Task Definition in AWS ECS. Thank you very much in advance!

jbm-composer commented 3 months ago

Did either of you work this out? I can't get past the above error (and I can't really figure out why it's being thrown anyway).

facebookresearch / audiocraft

Trying to run it on AWS using dora. #393