search

huawei-noah / SMARTS

Scalable Multi-Agent RL Training School for Autonomous Driving
MIT License
929 stars 186 forks source link

An error occurs when running Evaluation #159

Closed rsuwa closed 11 months ago

rsuwa commented 3 years ago

Reproduction

python evaluate.py scenarios/intersections/4lane -f agents/maddpg/baseline-lane-control.yaml --checkpoint ./log/results/run/4lane-4/MADDPG2_EarlyDone_c3c16_00000_0_2020-11-13_17-32-28/checkpoint_1200/checkpoint-1200

Error message

Traceback (most recent call last):
  File "evaluate.py", line 236, in <module>
    headless=args.headless,
  File "evaluate.py", line 114, in main
    trainer.restore(checkpoint)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/tune/trainable.py", line 476, in restore
    self.load_checkpoint(checkpoint_path)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 673, in load_checkpoint
    self.__setstate__(extra_data)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 164, in __setstate__
    Trainer.__setstate__(self, state)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 1168, in __setstate__
    self.workers.local_worker().restore(state["worker"])
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 969, in restore
    self.policy_map[pid].set_state(state)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/agents/maddpg/tf_policy.py", line 353, in set_state
    TFPolicy.set_state(self, state)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/policy/tf_policy.py", line 443, in set_state
    optimizer_vars = state.pop("_optimizer_variables", None)
TypeError: pop() takes at most 1 argument (2 given)

Environment

Python 3.7.9 OS: Ubuntu 18.04.5 LTS CPU: Ryzen 5 3600X GPU: GeForce RTX 2070 CUDA: 10.0

pip list

Package                  Version      Location
------------------------ ------------ ---------------------------------------------
absl-py                  0.9.0
aiohttp                  3.6.2
aiohttp-cors             0.7.0
aioredis                 1.3.1
apipkg                   1.5
astor                    0.8.1
async-timeout            3.0.1
atari-py                 0.2.6
attrs                    19.3.0
Automat                  20.2.0
beautifulsoup4           4.8.2
blessings                1.7
cachetools               4.1.1
certifi                  2020.6.20
chardet                  3.0.4
click                    7.1.1
cloudpickle              1.3.0
colorama                 0.4.3
colorful                 0.6.0a1
commonmark               0.9.1
constantly               15.1.0
coverage                 5.2.1
cycler                   0.10.0
Cython                   0.29.21
decorator                4.4.2
dm-tree                  0.1.2
execnet                  1.7.1
filelock                 3.0.12
future                   0.18.2
gast                     0.2.2
google                   3.0.0
google-api-core          1.23.0
google-auth              1.23.0
google-pasta             0.2.0
googleapis-common-protos 1.52.0
gpustat                  0.6.0
grpcio                   1.27.2
gym                      0.17.2
h5py                     2.10.0
hiredis                  1.1.0
hyperlink                20.0.1
idna                     2.9
imageio                  2.9.0
importlib-metadata       1.6.0
incremental              17.5.0
joblib                   0.14.1
jsonpatch                1.25
jsonpointer              2.0
jsonschema               3.2.0
Keras-Applications       1.0.8
Keras-Preprocessing      1.1.0
kiwisolver               1.2.0
lz4                      3.0.2
Markdown                 3.2.1
matplotlib               3.3.2
more-itertools           8.2.0
msgpack                  1.0.0
multidict                4.7.5
networkx                 2.4
numpy                    1.18.2
nvidia-ml-py3            7.352.0
opencensus               0.7.11
opencensus-context       0.1.2
opencv-python            4.2.0.32
opencv-python-headless   4.2.0.32
opt-einsum               3.2.0
packaging                20.4
panda3d                  1.10.6.post2
panda3d-gltf             0.6
panda3d-simplepbr        0.4
pandas                   1.0.3
Pillow                   7.0.0
pip                      20.2.2
pluggy                   0.13.1
prometheus-client        0.8.0
protobuf                 3.11.3
psutil                   5.7.0
py                       1.8.1
py-cpuinfo               5.0.0
py-spy                   0.3.3
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pybullet                 2.7.1
pyglet                   1.5.0
Pygments                 2.7.1
PyHamcrest               2.0.2
pynput                   1.6.8
pyparsing                2.4.6
pyrsistent               0.16.0
pytest                   5.4.1
pytest-benchmark         3.2.3
pytest-cov               2.10.0
pytest-forked            1.2.0
pytest-xdist             1.32.0
python-dateutil          2.8.1
python-xlib              0.27
pytz                     2019.3
PyWavelets               1.1.1
PyYAML                   5.3.1
pyzmq                    19.0.0
quadprog                 0.1.7
ray                      1.0.1
redis                    3.4.1
requests                 2.23.0
rich                     7.0.0
rsa                      4.6
Rtree                    0.9.4
scikit-image             0.17.2
scikit-learn             0.22.2.post1
scipy                    1.4.1
setuptools               49.6.0
sh                       1.12.14
Shapely                  1.7.0
six                      1.14.0
sklearn                  0.0
smarts                   0.4.3        /home/ryota/src/github.com/huawei-noah/SMARTS
soupsieve                2.0
supervisor               4.1.0
tableprint               0.9.0
tabulate                 0.8.7
tensorboard              1.15.0
tensorboardX             2.0
tensorflow-estimator     1.15.1
tensorflow-gpu           1.15.0
tensorflow-probability   0.8.0
termcolor                1.1.0
tifffile                 2020.9.3
torch                    1.3.0
torchfile                0.1.0
torchvision              0.4.1
tornado                  6.0.4
trimesh                  3.6.20
Twisted                  20.3.0
typing-extensions        3.7.4.3
urllib3                  1.25.8
visdom                   0.1.8.9
wcwidth                  0.1.9
websocket-client         0.57.0
Werkzeug                 1.0.0
wheel                    0.35.1
wrapt                    1.12.1
yarl                     1.4.2
yattag                   1.13.2
zipp                     3.1.0
zope.interface           5.1.0
KornbergFresnel commented 3 years ago

@rsuwa try ray==0.8.7. We gonna upgrade ray to v1.0 these days

rsuwa commented 3 years ago

@KornbergFresnel I tried ray==0.8.7 and got the same error.

kaixqu commented 3 years ago

Same here. What happened? Dict can have two parameters for pop, but list can only have one parameter. Is this some kind of bug from November 14, 2020?

ssexuejinwei commented 1 year ago

Same here. What happened? Did the checkpoint saved for maddpg have some bugs?

Adaickalavan commented 11 months ago

This issue is being closed as the marl_benchmark has been archived and is not expected to receive any further updates. See https://github.com/huawei-noah/SMARTS/pull/2025.