search

Tviskaron / pogema-baselines

PPO and PyMARL baseline for Pogema environment
20 stars 5 forks source link

Issue in running main.py #9

Closed rohitrajgopalan closed 2 weeks ago

rohitrajgopalan commented 12 months ago

Hi There,

I am trying to run the main.py file within the pymarl directory using the default parameter of configs\debug.yaml with Python 3.8.10 on Windows 11

However, I am encountering a "process died" message as shown below.

Here are the logs (I hope this is enough information for you to go by): PS C:\Users\rohit\Documents\pogema-baselines\pymarl> python .\main.py
[DEBUG 23:06:57] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:58] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:58] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=) [DEBUG 23:06:58] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:58] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:58] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=) [DEBUG 23:06:58] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:58] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:58] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=) [DEBUG 23:06:59] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:59] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:59] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=) [DEBUG 23:06:59] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:59] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=None) [DEBUG 23:06:59] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=C:\Users\rohit\Documents\pogema-baselines, universal_newlines=False, shell=None, istream=) usage: training_run.py [-h] [--config_path CONFIG_PATH] [--raw_config RAW_CONFIG] training_run.py: error: unrecognized arguments: {grid_config: {FREE: 0, OBSTACLE: 1, MOVES: [[0, 0], [-1, 0], [1, 0], [0, -1], [0, 1]], on_target: finish, seed: null, size: 8, density: 0.3, num_agents: 8, obs_radius: 5, agents_xy: n ull, targets_xy: null, collision_system: priority, persistent: false, observation_type: default, map: null, empty_outside: true, map_name: custom, integration: null, max_episode_steps: 64, auto_reset: null}}, agent_params: {agent: r nn, rnn_hidden_dim: 64, obs_agent_id: true, obs_last_action: true, repeat_id: 1, label: default_label}, rl_params: {gamma: 0.99, batch_size: 32, lr: 0.0005, critic_lr: 0.0005, optim_alpha: 0.99, optim_eps: 1e-05, grad_norm_clip: 10. 0}, logging: {use_tensorboard: true, save_model: true, save_model_interval: 10000000, checkpoint_path: , evaluate: false, load_step: 0, save_replay: false, local_results_path: results, use_wandb: false, project_name: Pogema-PyMARL-D ebug}, settings: {batch_size_run: 1, test_nepisode: 0, test_interval: 2000, test_greedy: true, log_interval: 2000, runner_log_interval: 2000, learner_log_interval: 2000, t_max: 10000, use_cuda: true, buffer_cpu_only: true, mac: basi c_mac, env: pogema}, algo_settings: {QMIX: {action_selector: epsilon_greedy, epsilon_start: 1.0, epsilon_finish: 0.05, epsilon_anneal_time: 50000, runner: episode, buffer_size: 5000, target_update_interval: 200, agent_output_type: q , learner: q_learner, double_q: true, name: qmix, mixer: qmix, mixing_embed_dim: 32, hypernet_layers: 2, hypernet_embed: 64}, VDN: {action_selector: epsilon_greedy, epsilon_start: 1.0, epsilon_finish: 0.05, epsilon_anneal_time: 5000 0, runner: episode, buffer_size: 5000, target_update_interval: 200, agent_output_type: q, learner: q_learner, double_q: true, name: vdn, mixer: vdn}, IQL: {action_selector: epsilon_greedy, epsilon_start: 1.0, epsilon_finish: 0.05, e psilon_anneal_time: 50000, runner: episode, buffer_size: 5000, target_update_interval: 200, agent_output_type: q, learner: q_learner, double_q: true, name: iql, mixer: null}, QTRAN: {action_selector: epsilon_greedy, epsilon_start: 1 .0, epsilon_finish: 0.05, epsilon_anneal_time: 50000, runner: episode, buffer_size: 5000, target_update_interval: 200, agent_output_type: q, learner: qtran_learner, double_q: true, name: qtran, mixer: qtran_base, mixing_embed_dim: 6 4, qtran_arch: qtran_paper, opt_loss: 1, nopt_min_loss: 0.1, network_size: small}, COMA: {action_selector: multinomial, epsilon_start: 0.5, epsilon_finish: 0.01, epsilon_anneal_time: 100000, runner: parallel, buffer_size: 1, target_update_interval: 200, agent_output_type: q, learner: q_learner, double_q: true, name: coma, mask_before_softmax: false, batch_size_run: 1, batch_size: 1, mixer: null}}, algo: QMIX}' [INFO 23:07:00] git.cmd Ignored error after process had died: OSError(9, 'The handle is invalid', None, 6, None) [INFO 23:07:00] git.cmd Ignored error after process had died: OSError(9, 'The handle is invalid', None, 6, None)

Tviskaron commented 12 months ago

Could you specify the versions of the libraries you are using? I don’t have a machine with Windows, but I'll attempt to reproduce your error on Linux. How did you install the dependencies? Have you tried using Docker? I'm also considering the possibility that the issue could be related to the versions of libraries required for PyMARL, as this library is somewhat outdated.

rohitrajgopalan commented 11 months ago

These are my list of dependencies I obtained from pip and no, I haven't tried using Docker

Package Version Editable project location

absl-py 1.4.0 addict 2.4.0 aiosignal 1.3.1 ale-py 0.8.1 appdirs 1.4.4 astunparse 1.6.3 asyncio 3.4.3 attrs 23.1.0 AutoROM 0.6.1 black 23.9.1 box2d-py 2.3.5 cachetools 5.3.0 casadi 3.6.3 certifi 2022.12.7 cffi 1.15.1 cfgv 3.3.1 charset-normalizer 3.0.1 click 8.1.3 clip 1.0 cloudpickle 2.2.1 colorama 0.4.6 confrez 0.1.0 contourpy 1.0.7 crazy-rl 0.2 cycler 0.11.0 data 0.4 decorator 5.1.1 distlib 0.3.6 docker-pycreds 0.4.0 docopt 0.6.2 einops 0.6.0 Farama-Notifications 0.0.4 filelock 3.9.0 fire 0.5.0 flatbuffers 23.3.3 flatdict 4.0.1 fonttools 4.38.0 frozenlist 1.4.0 ftfy 6.1.1 funcsigs 1.0.2 future 0.18.3 gast 0.4.0 geographiclib 2.0 geopy 2.3.0 gitdb 4.0.10 GitPython 3.1.31 google-auth 2.16.2 google-auth-oauthlib 1.0.0 google-pasta 0.2.0 grpcio 1.51.3 gym 0.26.2 gym-maze 0.4 gym-notices 0.0.8 gymnasium 0.29.1 gymnasium-notices 0.0.1 h5py 3.8.0 huggingface-hub 0.12.1 identify 2.5.22 idna 3.4 imageio 2.26.0 imgaug 0.4.0 importlib-resources 5.12.0 jax 0.4.18 jax-jumpy 1.0.0 jaxlib 0.4.18 joblib 1.2.0 jsonpickle 3.0.1 jsonschema 4.19.1 jsonschema-specifications 2023.7.1 keras 2.12.0 kiwisolver 1.4.4 lark 1.1.7 latex 0.7.0 lazy_loader 0.1 libclang 16.0.0 linear-rl 0.1 lmdb 1.4.0 Markdown 3.4.1 MarkupSafe 2.1.2 matplotlib 3.7.0 ml-dtypes 0.3.1 msgpack 1.0.7 munch 2.5.0 networkx 3.0 nodeenv 1.7.0 np 1.0.2 numpy 1.26.0 oauthlib 3.2.2 opencv-python 3.4.18.65 opt-einsum 3.3.0 packaging 23.0 pandas 2.1.1 parse 1.19.1 pathspec 0.11.2 pathtools 0.1.2 pettingzoo 1.24.1 pika 1.3.2 Pillow 10.0.1 pip 23.2.1 platformdirs 3.2.0 pogema 1.2.2 pre-commit 3.2.1 protobuf 4.22.0 psutil 5.9.4 py-cpuinfo 9.0.0 py4j 0.10.9.7 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycddlib 2.1.7 pycparser 2.21 pydantic 1.9.1 pyem 2.1.0 pygame 2.1.0 Pygments 2.16.1 pyhocon 0.3.60 pymunk 6.4.0 pyparsing 3.0.9 python-dateutil 2.8.2 pytope 0.0.4 pytz 2023.3.post1 PyWavelets 1.4.1 PyYAML 6.0 queue-env 0.1.0 c:\users\rohit\documents\smartnet-secondary\smartnet-scenario\experiment\queue_env ray 2.7.1 referencing 0.30.2 regex 2022.10.31 requests 2.28.2 requests-oauthlib 1.3.1 rich 13.6.0 rpds-py 0.10.6 rsa 4.9 sacred 0.8.4 scikit-image 0.20.0 scipy 1.10.1 seaborn 0.12.2 sentry-sdk 1.15.0 setproctitle 1.3.2 setuptools 67.4.0 shapely 2.0.1 Shimmy 0.2.1 shutilwhich 1.1.0 six 1.16.0 smartnet-tools 0.1.0 smmap 5.0.0 stable-baselines3 2.1.0 sumo-rl 1.4.3 C:\Users\rohit\Documents\sumo-rl sumolib 1.18.0 SuperSuit 3.3.0 swig 4.1.1 tabulate 0.9.0 tb-nightly 2.13.0a20230307 tempdir 0.7.1 tensorboard 2.12.2 tensorboard-data-server 0.7.0 tensorboard-logger 0.1.0 tensorboard-plugin-wit 1.8.1 tensorboardX 2.6.2.2 tensorflow 2.12.0 tensorflow-estimator 2.12.0 tensorflow-intel 2.12.0 tensorflow-io-gcs-filesystem 0.31.0 termcolor 2.3.0 tifffile 2023.2.28 timm 0.6.12 tomli 2.0.1 torch 1.13.1+cu117 torchaudio 0.13.1+cu117 torchvision 0.14.1+cu117 tqdm 4.65.0 traci 1.18.0 typer 0.9.0 typing_extensions 4.8.0 tzdata 2023.3 urllib3 1.26.14 virtualenv 20.21.0 wandb 0.14.0 wcwidth 0.2.6 Werkzeug 2.2.3 wheel 0.38.4 wrapt 1.14.1 yapf 0.32.0

Tviskaron commented 11 months ago

Based on what I can see, you are using versions that are different from those specified in docker/requirements.txt. I recommend trying with those specific versions. Additionally, I have been unable to reproduce your error on a Linux system.

By the way, the PyMARL code is somewhat outdated. I am planning to rewrite QMIX in the style of Clean RL, which will not only likely resolve your issue but also eliminate many dependencies. This should also address the memory leak problems discussed in this thread: https://github.com/Tviskaron/pogema-baselines/issues/10