`run_exp.sh` does not work

[HanYuanHsu]

I could not run the experiment by ./run_exp.sh qmix sc2_gen_terran. I got the following error.

root@03897ac7213a:/home/hanyuan/mappo# ./run_exp.sh qmix sc2_gen_terran
pygame 2.5.2 (SDL 2.28.2, Python 3.8.12)
Hello from the pygame community. https://www.pygame.org/contribute.html
No module named 'smac'
name 'StarCraft2CustomEnv' is not defined
[DEBUG 05:34:39] git.util Failed checking if running in CYGWIN due to: FileNotFoundError(2, 'No such file or directory')
[DEBUG 05:34:39] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=/home/hanyuan/mappo, stdin=None, shell=False, universal_newlines=False)
[DEBUG 05:34:39] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=/home/hanyuan/mappo, stdin=None, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=/home/hanyuan/mappo, stdin=<valid stream>, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=/home/hanyuan/mappo, stdin=None, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=/home/hanyuan/mappo, stdin=None, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=/home/hanyuan/mappo, stdin=<valid stream>, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=/home/hanyuan/mappo, stdin=None, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=/home/hanyuan/mappo, stdin=None, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=/home/hanyuan/mappo, stdin=<valid stream>, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=/home/hanyuan/mappo, stdin=None, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=/home/hanyuan/mappo, stdin=None, shell=False, universal_newlines=False)
[DEBUG 05:34:45] git.cmd Popen(['git', 'cat-file', '--batch-check'], cwd=/home/hanyuan/mappo, stdin=<valid stream>, shell=False, universal_newlines=False)
src/main.py:91: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
  if isinstance(v, collections.Mapping):
[INFO 05:34:46] root Saving to FileStorageObserver in results/sacred.
Traceback (most recent call last):
  File "src/main.py", line 145, in <module>
    ex.run_commandline(params)
  File "/opt/conda/lib/python3.8/site-packages/sacred-0.7.2-py3.8.egg/sacred/experiment.py", line 250, in run_commandline
    return self.run(cmd_name, config_updates, named_configs, {}, args)
  File "/opt/conda/lib/python3.8/site-packages/sacred-0.7.2-py3.8.egg/sacred/experiment.py", line 197, in run
    run = self._create_run(command_name, config_updates, named_configs,
  File "/opt/conda/lib/python3.8/site-packages/sacred-0.7.2-py3.8.egg/sacred/experiment.py", line 421, in _create_run
    run = create_run(self, command_name, config_updates,
  File "/opt/conda/lib/python3.8/site-packages/sacred-0.7.2-py3.8.egg/sacred/initialize.py", line 378, in create_run
    scaffold.finalize_initialization(run=run)
  File "/opt/conda/lib/python3.8/site-packages/sacred-0.7.2-py3.8.egg/sacred/initialize.py", line 152, in finalize_initialization
    self._warn_about_suspicious_changes()
  File "/opt/conda/lib/python3.8/site-packages/sacred-0.7.2-py3.8.egg/sacred/initialize.py", line 157, in _warn_about_suspicious_changes
    raise KeyError('Added a new config entry "{}" that is not used'
KeyError: 'Added a new config entry "clip_range" that is not used anywhere'

Here are the setups I have done before running the command above: First, I moved the Dockerfile from the docker folder to . and modified it to the following:

FROM pytorch/pytorch:1.11.0-cuda11.3-cudnn8-devel
MAINTAINER Christian Schroeder de Witt

RUN rm /etc/apt/sources.list.d/cuda.list
RUN rm /etc/apt/sources.list.d/nvidia-ml.list

# added install wget unzip nano
RUN apt-get update && apt-get install -y git && \
    apt update && apt install wget unzip nano

RUN pip install --upgrade pip
RUN pip install numpy scipy pyyaml matplotlib imageio pygame tensorboard-logger ruamel.base ryd wandb

# added mkdir /root/pymarl
RUN mkdir /install && mkdir /root/pymarl
WORKDIR /install

RUN pip install jsonpickle==0.9.6
# install Sacred (from OxWhirl fork)
RUN pip install setuptools
RUN git clone https://github.com/oxwhirl/sacred.git /install/sacred && cd /install/sacred && python setup.py install

# Install pymongo
RUN pip install pymongo

## -- SMAC
# changed SC2PATH
ENV smac_ver 1
RUN pip install "protobuf<3.21" git+https://github.com/oxwhirl/smacv2.git
ENV SC2PATH /root/pymarl/3rdparty/StarCraftII

RUN git clone https://github.com/koulanurag/ma-gym.git /install/ma-gym && cd /install/ma-gym && pip install -e .

RUN useradd -u 1000 -m hanyuan && \
    mkdir /home/hanyuan/mappo

WORKDIR /home/hanyuan/mappo
COPY . /home/hanyuan/mappo

Then, I ran docker build -t pymarl:smacv2 . and docker run -it pymarl:smacv2 /bin/bash to enter the container. In the container, I ran ./install_sc2.sh to set up StarCraft II, which should be successful because when I ran ls $SC2PATH/Maps, it showed

Ladder2017Season2  Ladder2017Season4  Ladder2018Season2  Ladder2018Season4  Melee      mini_games
Ladder2017Season1  Ladder2017Season3  Ladder2018Season1  Ladder2018Season3  Ladder2019Season1  SMAC_Maps

Then, I copied 32x32_flat.SC2Map to $SC2PATH/Maps.

Finally, I set use_wandb=False in the config file within the container.

These were the steps I have done before running ./run_exp.sh qmix sc2_gen_terran and getting the error mentioned in the beginning. I do not understand the part 'Added a new config entry "clip_range" that is not used anywhere' and no module named 'smac'.

[benellis3]

You seem to be trying to run qmix with this repo. This will not work. This repo only has working code for MAPPO. To reproduce the QMIX experiments, look at this repo: https://github.com/benellis3/pymarl2

[HanYuanHsu]

Thank you. I will try MAPPO instead. By the way, am I setting up the environment correctly?