search

google-research / reincarnating_rl

[NeurIPS 2022] Open source code for reusing prior computational work in RL.
https://agarwl.github.io/reincarnating_rl
Apache License 2.0
91 stars 12 forks source link

Run script problem #1

Open sdpkjc opened 1 year ago

sdpkjc commented 1 year ago

Run command in README

cd reincarnating_rl
python -um reincarnating_rl.train \
  --agent qdagger_rainbow \
  --gin_files reincarnating_rl/configs/qdagger_rainbow.gin
  --base_dir /tmp/qdagger_rainbow \
  --teacher_checkpoint_dir $TEACHER_CKPT_DIR/Breakout/1 \
  --teacher_checkpoint_number 399
  --run_number=1 \
  --atari_roms_path=/tmp/atari_roms \
  --alsologtostderr

I initially modified the command as follows:

cd reincarnating_rl
python -um reincarnating_rl.train \
  --agent qdagger_rainbow \
  --gin_files reincarnating_rl/configs/qdagger_rainbow.gin \
  --base_dir /tmp/qdagger_rainbow \
  --teacher_checkpoint_dir $TEACHER_CKPT_DIR/Breakout/1 \
  --teacher_checkpoint_number 399 \
  --run_number=1 \
  --alsologtostderr

But the system reported an error, I hope to get your help. 🙏

2023-03-07 14:27:23.087431: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-03-07 14:27:23.178229: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/cv2/../../lib64:
2023-03-07 14:27:23.178255: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/tensorflow/python/framework/dtypes.py:246: DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`.  (Deprecated NumPy 1.24)
  np.bool8: (False, True),
2023-03-07 14:27:23.629774: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/cv2/../../lib64:
2023-03-07 14:27:23.629827: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/cv2/../../lib64:
2023-03-07 14:27:23.629835: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/tf/__init__.py:48: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if (distutils.version.LooseVersion(tf.__version__) <
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/jax/_src/deprecations.py:51: DeprecationWarning: jax.interpreters.xla.DeviceArray is deprecated. Use jax.Array instead.
  warnings.warn(message, DeprecationWarning)
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/jax/_src/api_util.py:240: SyntaxWarning: Jitted function has invalid argnames {'distill_temperature', 'distill_type', 'distill_loss_coefficient', 'distill_best_action_only'} in static_argnames. Function does not take these args.This warning will be replaced by an error after 2022-08-20 at the earliest.
  warnings.warn(f"Jitted function has invalid argnames {invalid_argnames} "
2023-03-07 14:27:24.751736: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/cv2/../../lib64:
2023-03-07 14:27:24.751770: W tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:265] failed call to cuInit: UNKNOWN ERROR (303)
2023-03-07 14:27:24.751785: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (sdpkjc008-B460MAORUSPRO): /proc/driver/nvidia/version does not exist
I0307 14:27:24.751885 140636716074048 train.py:126] Setting random seed: 1
Traceback (most recent call last):
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/train.py", line 174, in <module>
    app.run(main)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/train.py", line 151, in main
    base_run_experiment.load_gin_configs(gin_files, gin_bindings)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/dopamine/discrete_domains/run_experiment.py", line 55, in load_gin_configs
    gin.parse_config_files_and_bindings(gin_files,
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 2497, in parse_config_files_and_bindings
    includes_and_imports = parse_config_file(config_file, skip_unknown)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 2450, in parse_config_file
    includes, imports = parse_config(f, skip_unknown=skip_unknown)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 2322, in parse_config
    for statement in parser:
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config_parser.py", line 211, in __next__
    statement = self.parse_statement()
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config_parser.py", line 240, in parse_statement
    value = self.parse_value()
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config_parser.py", line 280, in parse_value
    success, value = parser()
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config_parser.py", line 558, in _maybe_parse_configurable_reference
    reference = self._delegate.configurable_reference(scoped_name, evaluate)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 60, in try_with_location
    augment_exception_message_and_reraise(exception, _format_location(location))
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 56, in try_with_location
    yield
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config_parser.py", line 558, in _maybe_parse_configurable_reference
    reference = self._delegate.configurable_reference(scoped_name, evaluate)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 844, in configurable_reference
    return ConfigurableReference(scoped_selector, evaluate)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 691, in __init__
    self.initialize()
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 697, in initialize
    _raise_unknown_reference_error(self)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 644, in _raise_unknown_reference_error
    raise ValueError(err_str.format(ref.selector, maybe_parens, additional_msg))
ValueError: No configurable matching reference '@loss_helpers.persistence_linearly_decaying_epsilon'.
  In file "reincarnating_rl/configs/qdagger_rainbow.gin", line 27
    JaxFullRainbowAgent.epsilon_fn = @loss_helpers.persistence_linearly_decaying_epsilon
                                     ^

My system info:

agarwl commented 1 year ago

So, the gin file should be modified to JaxFullRainbowAgent.epsilon_fn = @loss_helpers.reincarnation_linearly_decaying_epsilon. Can you maybe also send a pull request with these changes?

sdpkjc commented 1 year ago

Thank for you reply!

I modified JaxFullRainbowAgent.epsilon_fn in file reincarnating_rl/configs/qdagger_drq.gin and file reincarnating_rl/configs/qdagger_rainbow.gin to @loss_helpers.reincarnation_linearly_decaying_epsilon, but it still didn't work.

Error Infos:

2023-03-10 09:52:30.681166: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-03-10 09:52:30.779437: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/cv2/../../lib64:
2023-03-10 09:52:30.779487: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/tensorflow/python/framework/dtypes.py:246: DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`.  (Deprecated NumPy 1.24)
  np.bool8: (False, True),
2023-03-10 09:52:31.232661: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/cv2/../../lib64:
2023-03-10 09:52:31.232714: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/cv2/../../lib64:
2023-03-10 09:52:31.232723: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/tf/__init__.py:48: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if (distutils.version.LooseVersion(tf.__version__) <
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/jax/_src/api_util.py:240: SyntaxWarning: Jitted function has invalid argnames {'distill_type', 'distill_temperature', 'distill_loss_coefficient', 'distill_best_action_only'} in static_argnames. Function does not take these args.This warning will be replaced by an error after 2022-08-20 at the earliest.
  warnings.warn(f"Jitted function has invalid argnames {invalid_argnames} "
2023-03-10 09:52:32.407890: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/cv2/../../lib64:
2023-03-10 09:52:32.407952: W tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:265] failed call to cuInit: UNKNOWN ERROR (303)
2023-03-10 09:52:32.407987: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (sdpkjc008-B460MAORUSPRO): /proc/driver/nvidia/version does not exist
I0310 09:52:32.408150 140470457111616 train.py:126] Setting random seed: 1
W0310 09:52:32.412673 140470457111616 run_experiment.py:267] DEPRECATION WARNING: Logger is being deprecated. Please switch to CollectorDispatcher!
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gym/envs/registration.py:593: UserWarning: WARN: The environment PongNoFrameskip-v0 is out of date. You should consider upgrading to version `v4`.
  logger.warn(
A.L.E: Arcade Learning Environment (version 0.7.5+db37282)
[Powered by Stella]
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gym/core.py:317: DeprecationWarning: WARN: Initializing wrapper in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
  deprecation(
/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gym/wrappers/step_api_compatibility.py:39: DeprecationWarning: WARN: Initializing environment in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
  deprecation(
I0310 09:52:32.514550 140470457111616 reincarnation_rainbow_agent.py:43] Creating ReincarnationRainbowAgent agent with the following parameters:
I0310 09:52:32.514734 140470457111616 reincarnation_rainbow_agent.py:45]     num_updates_per_persistent_step: 1
I0310 09:52:32.514835 140470457111616 reincarnation_rainbow_agent.py:47]     data_augmentation: False
I0310 09:52:32.514869 140470457111616 reincarnation_rainbow_agent.py:48]     network: <class 'dopamine.jax.networks.FullRainbowNetwork'>
I0310 09:52:32.515293 140470457111616 full_rainbow_agent.py:224] Creating QDaggerRainbowAgent agent with the following parameters:
I0310 09:52:32.515471 140470457111616 full_rainbow_agent.py:226]     double_dqn: False
I0310 09:52:32.515506 140470457111616 full_rainbow_agent.py:227]     noisy_networks: False
I0310 09:52:32.515537 140470457111616 full_rainbow_agent.py:228]     dueling_dqn: False
I0310 09:52:32.515613 140470457111616 full_rainbow_agent.py:229]     distributional: True
I0310 09:52:32.515684 140470457111616 full_rainbow_agent.py:230]     mse_loss: 0
I0310 09:52:32.515713 140470457111616 full_rainbow_agent.py:231]     num_atoms: 51
I0310 09:52:32.515762 140470457111616 full_rainbow_agent.py:232]     replay_scheme: prioritized
I0310 09:52:32.515809 140470457111616 full_rainbow_agent.py:233]     num_updates_per_train_step: 1
I0310 09:52:32.525550 140470457111616 xla_bridge.py:413] Unable to initialize backend 'tpu_driver': NOT_FOUND: Unable to find driver in registry given worker: 
I0310 09:52:32.525683 140470457111616 xla_bridge.py:413] Unable to initialize backend 'cuda': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'
I0310 09:52:32.525734 140470457111616 xla_bridge.py:413] Unable to initialize backend 'rocm': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'
I0310 09:52:32.526005 140470457111616 xla_bridge.py:413] Unable to initialize backend 'tpu': INVALID_ARGUMENT: TpuPlatform is not available.
I0310 09:52:32.526068 140470457111616 xla_bridge.py:413] Unable to initialize backend 'plugin': xla_extension has no attributes named get_plugin_device_client. Compile TensorFlow with //tensorflow/compiler/xla/python:enable_plugin_device set to true (defaults to false) to enable this.
W0310 09:52:32.526121 140470457111616 xla_bridge.py:420] No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
I0310 09:52:32.544177 140470457111616 dqn_agent.py:278] Creating QDaggerRainbowAgent agent with the following parameters:
I0310 09:52:32.544276 140470457111616 dqn_agent.py:280]      gamma: 0.990000
I0310 09:52:32.544321 140470457111616 dqn_agent.py:281]      update_horizon: 3.000000
I0310 09:52:32.544353 140470457111616 dqn_agent.py:282]      min_replay_history: 20000
I0310 09:52:32.544383 140470457111616 dqn_agent.py:283]      update_period: 4
I0310 09:52:32.544418 140470457111616 dqn_agent.py:284]      target_update_period: 8000
I0310 09:52:32.544449 140470457111616 dqn_agent.py:285]      epsilon_train: 0.010000
I0310 09:52:32.544476 140470457111616 dqn_agent.py:286]      epsilon_eval: 0.001000
I0310 09:52:32.544504 140470457111616 dqn_agent.py:287]      epsilon_decay_period: 50000
I0310 09:52:32.544531 140470457111616 dqn_agent.py:288]      optimizer: adam
I0310 09:52:32.544557 140470457111616 dqn_agent.py:289]      seed: 1
I0310 09:52:32.544584 140470457111616 dqn_agent.py:290]      loss_type: mse
I0310 09:52:32.544611 140470457111616 dqn_agent.py:291]      preprocess_fn: <function preprocess_inputs_with_augmentation at 0x7fc1235a1e50>
I0310 09:52:32.544739 140470457111616 dqn_agent.py:292]      summary_writing_frequency: 25000
I0310 09:52:32.544775 140470457111616 dqn_agent.py:293]      allow_partial_reload: False
2023-03-10 09:52:32.545303: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
I0310 09:52:32.600445 140470457111616 circular_replay_buffer.py:157] Creating a OutOfGraphPrioritizedReplayBuffer replay memory with the following parameters:
I0310 09:52:32.600550 140470457111616 circular_replay_buffer.py:160]     observation_shape: (84, 84)
I0310 09:52:32.600592 140470457111616 circular_replay_buffer.py:161]     observation_dtype: <class 'jax.numpy.uint8'>
I0310 09:52:32.600625 140470457111616 circular_replay_buffer.py:162]     terminal_dtype: <class 'numpy.uint8'>
I0310 09:52:32.600655 140470457111616 circular_replay_buffer.py:163]     stack_size: 4
I0310 09:52:32.600683 140470457111616 circular_replay_buffer.py:164]     replay_capacity: 1000000
I0310 09:52:32.600711 140470457111616 circular_replay_buffer.py:165]     batch_size: 32
I0310 09:52:32.600739 140470457111616 circular_replay_buffer.py:166]     update_horizon: 3
I0310 09:52:32.600765 140470457111616 circular_replay_buffer.py:167]     gamma: 0.990000
I0310 09:52:32.600794 140470457111616 circular_replay_buffer.py:168]     checkpoint_duration: 4
I0310 09:52:32.600822 140470457111616 circular_replay_buffer.py:169]     keep_every: None
I0310 09:52:33.608599 140470457111616 loss_helpers.py:83] Creating Adam optimizer with settings lr=0.000063, beta1=0.900000, beta2=0.999000, eps=0.000150
Traceback (most recent call last):
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/train.py", line 174, in <module>
    app.run(main)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/train.py", line 162, in main
    runner = runner_fn(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/run_experiment.py", line 301, in __init__
    super().__init__(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/run_experiment.py", line 76, in __init__
    super().__init__(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/dopamine/discrete_domains/run_experiment.py", line 229, in __init__
    self._agent = create_agent_fn(self._sess, self._environment,
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/train.py", line 103, in create_agent
    return agent_fn(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/qdagger_rainbow_agent.py", line 153, in __init__
    super().__init__(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/reincarnation_rainbow_agent.py", line 50, in __init__
    super().__init__(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/dopamine/labs/atari_100k/atari_100k_rainbow_agent.py", line 106, in __init__
    super().__init__(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/dopamine/jax/agents/full_rainbow/full_rainbow_agent.py", line 248, in __init__
    super().__init__(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/dopamine/jax/agents/dqn/dqn_agent.py", line 336, in __init__
    self._build_networks_and_optimizer()
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/qdagger_rainbow_agent.py", line 186, in _build_networks_and_optimizer
    self.optimizer = loss_helpers.create_pretraining_optimizer(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1605, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
    raise proxy.with_traceback(exception.__traceback__) from None
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/gin/config.py", line 1582, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/home/sdpkjc008/dev/reincarnating_rl/reincarnating_rl/loss_helpers.py", line 85, in create_pretraining_optimizer
    return wrapper_fn(optax.adam)(
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/site-packages/optax/_src/schedule.py", line 579, in wrapped_transform
    elif isinstance(value, (int, float, chex.Array)):
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/typing.py", line 720, in __instancecheck__
    return self.__subclasscheck__(type(obj))
  File "/home/sdpkjc008/anaconda3/envs/rrl/lib/python3.9/typing.py", line 723, in __subclasscheck__
    raise TypeError("Subscripted generics cannot be used with"
TypeError: Subscripted generics cannot be used with class and instance checks
  In call to configurable 'create_pretraining_optimizer' (<function create_pretraining_optimizer at 0x7fc123590670>)
  In call to configurable 'JaxDQNAgent' (<class 'dopamine.jax.agents.dqn.dqn_agent.JaxDQNAgent'>)
  In call to configurable 'JaxFullRainbowAgent' (<class 'dopamine.jax.agents.full_rainbow.full_rainbow_agent.JaxFullRainbowAgent'>)
  In call to configurable 'Atari100kRainbowAgent' (<class 'dopamine.labs.atari_100k.atari_100k_rainbow_agent.Atari100kRainbowAgent'>)
  In call to configurable 'ReincarnationRainbowAgent' (<class 'reincarnating_rl.reincarnation_rainbow_agent.ReincarnationRainbowAgent'>)
  In call to configurable 'QDaggerRainbowAgent' (<class 'reincarnating_rl.qdagger_rainbow_agent.QDaggerRainbowAgent'>)
  In call to configurable 'Runner' (<class 'dopamine.discrete_domains.run_experiment.Runner'>)
  In call to configurable 'RunnerWithTeacher' (<class 'reincarnating_rl.run_experiment.RunnerWithTeacher'>)
  In call to configurable 'ReincarnationRunner' (<class 'reincarnating_rl.run_experiment.ReincarnationRunner'>)
agarwl commented 1 year ago

I haven't seen this error before either -- seems like an issue to the arguments of the function create_pretraining_optimizer?