search

Improbable-AI / curiosity_baselines

An open source reinforcement learning codebase with a variety of intrinsic exploration methods implemented in PyTorch.
MIT License
10 stars 4 forks source link

namedtuple class not found error in `info_to_nt` #1

Open Animadversio opened 2 years ago

Animadversio commented 2 years ago

Hi I'm trying out this library on Windows10. There is a weird bug that I encounter when trying to launch a training with the following trianing code. 

python launch.py -alg ppo -curiosity_alg icm -env DeepmindMaze-v0 -iterations 100000000 -lstm -num_envs 4 -sample_mode cpu -num_gpus 0 -num_cpus 1 -eval_envs 0 -eval_max_steps 51000 -eval_max_traj 50 -timestep_limit 20 -log_interval 10000 -record_freq 0 -pretrain None -log_dir results/ppo_DeepmindMaze-v0/run_0 -discount 0.99 -lr 0.0001 -v_loss_coeff 1.0 -entropy_loss_coeff 0.001 -grad_norm_bound 1.0 -gae_lambda 0.95 -minibatches 1 -epochs 3 -ratio_clip 0.1 -kernel_mu 0.0 -kernel_sigma 0.001 -obs_type mask -max_episode_steps 500 -feature_encoding idf_burda -forward_loss_wt 0.2 -prediction_beta 1.0 -prediction_lr_scale 10.0 -launch_tmux no

The core error happens within info_to_nt 

2022-04-03 21:31:26.371369  | ppo_DeepmindMaze-v0_0 Runner  master CPU affinity: [0, 1, 2, 3, 4, 5, 6, 7].
2022-04-03 21:31:26.372369  | ppo_DeepmindMaze-v0_0 Runner  master Torch threads: 4.
using seed 5338
<__array_function__ internals>:5: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
<__array_function__ internals>:5: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
Process Process-2:
Traceback (most recent call last):
  File "C:\Users\binxu\.conda\envs\rl\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Users\binxu\.conda\envs\rl\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "E:\DL_Projects\RL\curiosity_baselines\rlpyt\samplers\buffer.py", line 76, in get_example_outputs
    o, r, d, env_info = env.step(action)
  File "E:\DL_Projects\RL\curiosity_baselines\rlpyt\envs\gym.py", line 75, in step
    info = info_to_nt(info)
  File "E:\DL_Projects\RL\curiosity_baselines\rlpyt\envs\gym.py", line 115, in info_to_nt
    ntc = globals()[name]
KeyError: 'info'

My understanding is that there shall be a namedtuple class called info defined before in globals() with build_info_tuples function . However, there is no such class defined .... I'm not sure what could have gone wrong. Something related to subprocessing in windows? or something else?

Is there ways to circumvent the info_to_nt

My python version is 3.8.5 torch.version== '1.10.2'

Thanks in advance!

Animadversio commented 2 years ago

One hypothesis is that the rlpyt.envs.gym.info class is defined in the main thread however it's not accessible in the subprocess. Similarly the info class defined in subprocess is not accessible in the main process...

Animadversio commented 2 years ago

I think this problem is related to the known issue in the rlpyt library... One suggestion they made is to substitute dynamically generated namedtuple type with NamedTupleSchema .. https://github.com/astooke/rlpyt/issues/99#issue-553094063 https://github.com/astooke/rlpyt/issues/168#issue-641535867