search

marl-book / codebase

Official code repo for the MARL book (www.marl-book.com)
385 stars 58 forks source link

Getting error when try to run the code. #29

Open Jaron-U opened 1 month ago

Jaron-U commented 1 month ago

Hi,

I got this error when i tried to run the code:

HYDRA_FULL_ERROR=1 python run.py +algorithm=ac env.name="lbforaging:Foraging-8x8-2p-3f-v3" env.time_limit=25
(15698) [WARNING] - (10/21 18:02:57) - root >> No seed has been set.
ActorCritic(
  (actor): MultiAgentFCNetwork(
    (independent): ModuleList(
      (0-1): 2 x Sequential(
        (0): Linear(in_features=15, out_features=64, bias=True)
        (1): ReLU()
        (2): Linear(in_features=64, out_features=64, bias=True)
        (3): ReLU()
        (4): Linear(in_features=64, out_features=6, bias=True)
      )
    )
  )
  (critic): MultiAgentFCNetwork(
    (independent): ModuleList(
      (0-1): 2 x Sequential(
        (0): Linear(in_features=15, out_features=64, bias=True)
        (1): ReLU()
        (2): Linear(in_features=64, out_features=64, bias=True)
        (3): ReLU()
        (4): Linear(in_features=64, out_features=1, bias=True)
      )
    )
  )
  (target_critic): MultiAgentFCNetwork(
    (independent): ModuleList(
      (0-1): 2 x Sequential(
        (0): Linear(in_features=15, out_features=64, bias=True)
        (1): ReLU()
        (2): Linear(in_features=64, out_features=64, bias=True)
        (3): ReLU()
        (4): Linear(in_features=64, out_features=1, bias=True)
      )
    )
  )
)
/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py:245: UserWarning: WARN: The reward returned by `step()` must be a float, int, np.integer or np.floating, actual type: <class 'list'>
  logger.warn(
/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py:245: UserWarning: WARN: The reward returned by `step()` must be a float, int, np.integer or np.floating, actual type: <class 'list'>
  logger.warn(
/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py:245: UserWarning: WARN: The reward returned by `step()` must be a float, int, np.integer or np.floating, actual type: <class 'list'>
  logger.warn(
/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py:245: UserWarning: WARN: The reward returned by `step()` must be a float, int, np.integer or np.floating, actual type: <class 'list'>
  logger.warn(
/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py:245: UserWarning: WARN: The reward returned by `step()` must be a float, int, np.integer or np.floating, actual type: <class 'list'>
  logger.warn(
/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py:245: UserWarning: WARN: The reward returned by `step()` must be a float, int, np.integer or np.floating, actual type: <class 'list'>
  logger.warn(
/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py:245: UserWarning: WARN: The reward returned by `step()` must be a float, int, np.integer or np.floating, actual type: <class 'list'>
  logger.warn(
/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/gymnasium/utils/passive_env_checker.py:245: UserWarning: WARN: The reward returned by `step()` must be a float, int, np.integer or np.floating, actual type: <class 'list'>
  logger.warn(
Error executing job with overrides: ['+algorithm=ac', 'env.name=lbforaging:Foraging-8x8-2p-3f-v3', 'env.time_limit=25']
Traceback (most recent call last):
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
    return _target_(*args, **kwargs)
  File "/home/jianglongyu/Documents/multi_agent/marl_lbforaging/marlbase/ac/train.py", line 103, in main
    batch_rew[n, :] = torch.tensor(reward)
RuntimeError: The expanded size of the tensor (2) must match the existing size (8) at non-singleton dimension 1.  Target sizes: [8, 2].  Tensor sizes: [8]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jianglongyu/Documents/multi_agent/marl_lbforaging/marlbase/run.py", line 43, in <module>
    main()
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/main.py", line 94, in decorated_main
    _run_hydra(
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
    raise ex
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
    return func()
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/home/jianglongyu/Documents/multi_agent/marl_lbforaging/marlbase/run.py", line 37, in main
    hydra.utils.call(cfg.algorithm, env, eval_env, logger, _recursive_=False)
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 226, in instantiate
    return instantiate_node(
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 347, in instantiate_node
    return _call_target(_target_, partial, args, kwargs, full_key)
  File "/home/jianglongyu/miniconda3/envs/marlbase/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target
    raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error in call to target 'ac.train.main':
RuntimeError('The expanded size of the tensor (2) must match the existing size (8) at non-singleton dimension 1.  Target sizes: [8, 2].  Tensor sizes: [8]')
full_key: algorithm
Jaron-U commented 1 month ago

update: I can run the dqn

LukasSchaefer commented 3 days ago

Hi! I just had a look at this and I am unable to reproduce your error. It appears you are using an older version of the repo (since algorithm name ac has been deprecated and should be replaced with ia2c). Please update the repo to the most recent version.

Also, a recent update of the gymnasium library causes issues with assumptions made in our code since they changed their interface for vectorised environments (hence also why only AC algorithms are affected), see here for more details. As a simple fix for now, the requirements.txt was updated to require a version of gymnasium that is preceding the newest 1.0.0 release. Please make sure you update/ downgrade the repo and the gymnasium library accordingly and let me know if that resolves your issues!