Open manuel-delverme opened 3 years ago
/home/esac/projects/venv/bin/python /home/esac/projects/XXXX/train.py experiment_id: [CLUSTER] baseline HEAD is now at 9c07f16 Merge pull request #68 from XXXX To github.com:XXXX.git * [new tag] snapshot/main/1098790182838c62f081944151c0c29ea026b4ae -> snapshot/main/1098790182838c62f081944151c0c29ea026b4ae Switched to branch 'main' monitor your run on https://wandb.ai/ /tmp/experiment_buddy-zCE8iXKnF4 Slurmctld(primary) at slurm is UP Slurmctld(backup) at slurmctl is DOWN bash -l /tmp/experiment_buddy-zCE8iXKnF4/run_experiment.sh git@github.com:XXXX.git train.py 1098790182838c62f081944151c0c29ea026b4ae 0%| | 0/1 [00:00<?, ?it/s][DEPLOY LOG] Refreshing modules... The following modules were not unloaded: (Use "module --force purge" to unload all): 1) gcc/7.4.0 2) Mila [=== Module python/3.7 loaded ===] [DEPLOY LOG] script realpath: /tmp/experiment_buddy-zCE8iXKnF4/run_experiment.sh [DEPLOY LOG] scripts home: /tmp/experiment_buddy-zCE8iXKnF4 [DEPLOY LOG] cd /home/mila/d/delvermm/experiments/ [DEPLOY LOG] EXPERIMENT_FOLDER=./tmp.J4te8WiRjV [DEPLOY LOG] downloading source code from git@github.com:XXXX.git to ./tmp.J4te8WiRjV Cloning into './tmp.J4te8WiRjV'... Checking out files: 100% (56/56), done. Note: checking out '1098790182838c62f081944151c0c29ea026b4ae'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new-branch-name> HEAD is now at 1098790 [CLUSTER] baseline [DEPLOY LOG] pwd is now /home/mila/d/delvermm/experiments/tmp.J4te8WiRjV [DEPLOY LOG] Using shared venv @ /home/mila/d/delvermm/venv Requirement already satisfied: pip in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (21.1.1) [DEPLOY LOG] installing experiment_buddy Obtaining experiment_buddy from git+https://github.com/ministry-of-silly-code/experiment_buddy#egg=experiment_buddy Updating /home/mila/d/delvermm/venv/src/experiment-buddy clone Running command git fetch -q --tags Running command git reset --hard -q 8152622bea8f8dd5632797698ac564a3f6b0c30e Requirement already satisfied: GitPython in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (3.1.14) Requirement already satisfied: tensorboardX in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (2.2) Requirement already satisfied: matplotlib in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (3.4.1) Requirement already satisfied: wandb in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (0.10.24) Requirement already satisfied: fabric in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (2.6.0) Requirement already satisfied: cloudpickle in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (1.2.2) Requirement already satisfied: PyYaml in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (5.4.1) Requirement already satisfied: paramiko in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (2.7.2) Requirement already satisfied: tqdm in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy) (4.59.0) Requirement already satisfied: invoke<2.0,>=1.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from fabric->experiment_buddy) (1.5.0) Requirement already satisfied: pathlib2 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from fabric->experiment_buddy) (2.3.5) Requirement already satisfied: bcrypt>=3.1.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from paramiko->experiment_buddy) (3.2.0) Requirement already satisfied: pynacl>=1.0.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from paramiko->experiment_buddy) (1.4.0) Requirement already satisfied: cryptography>=2.5 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from paramiko->experiment_buddy) (3.4.7) Requirement already satisfied: cffi>=1.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko->experiment_buddy) (1.14.5) Requirement already satisfied: six>=1.4.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko->experiment_buddy) (1.15.0) Requirement already satisfied: pycparser in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko->experiment_buddy) (2.20) Requirement already satisfied: gitdb<5,>=4.0.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from GitPython->experiment_buddy) (4.0.7) Requirement already satisfied: smmap<5,>=3.0.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from gitdb<5,>=4.0.1->GitPython->experiment_buddy) (4.0.0) Requirement already satisfied: python-dateutil>=2.7 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->experiment_buddy) (2.8.1) Requirement already satisfied: kiwisolver>=1.0.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->experiment_buddy) (1.3.1) Requirement already satisfied: pyparsing>=2.2.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->experiment_buddy) (2.4.7) Requirement already satisfied: cycler>=0.10 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->experiment_buddy) (0.10.0) Requirement already satisfied: pillow>=6.2.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->experiment_buddy) (7.2.0) Requirement already satisfied: numpy>=1.16 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->experiment_buddy) (1.20.2) Requirement already satisfied: protobuf>=3.8.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboardX->experiment_buddy) (3.15.7) Requirement already satisfied: promise<3,>=2.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (2.3) Requirement already satisfied: pathtools in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (0.1.2) Requirement already satisfied: sentry-sdk>=0.4.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (1.0.0) Requirement already satisfied: requests<3,>=2.0.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (2.25.1) Requirement already satisfied: subprocess32>=3.5.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (3.5.4) Requirement already satisfied: docker-pycreds>=0.4.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (0.4.0) Requirement already satisfied: configparser>=3.8.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (5.0.2) Requirement already satisfied: shortuuid>=0.5.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (1.0.1) Requirement already satisfied: Click>=7.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (7.1.2) Requirement already satisfied: psutil>=5.0.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->experiment_buddy) (5.8.0) Requirement already satisfied: idna<3,>=2.5 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from requests<3,>=2.0.0->wandb->experiment_buddy) (2.10) Requirement already satisfied: certifi>=2017.4.17 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from requests<3,>=2.0.0->wandb->experiment_buddy) (2020.12.5) Requirement already satisfied: chardet<5,>=3.0.2 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from requests<3,>=2.0.0->wandb->experiment_buddy) (3.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from requests<3,>=2.0.0->wandb->experiment_buddy) (1.26.4) Installing collected packages: experiment-buddy Attempting uninstall: experiment-buddy Found existing installation: experiment-buddy 0.0.1 Uninstalling experiment-buddy-0.0.1: Successfully uninstalled experiment-buddy-0.0.1 Running setup.py develop for experiment-buddy Successfully installed experiment-buddy-0.0.1 Looking in links: https://download.pytorch.org/whl/torch_stable.html Obtaining experiment_buddy from git+https://github.com/ministry-of-silly-code/experiment_buddy#egg=experiment_buddy (from -r requirements.txt (line 1)) Updating /home/mila/d/delvermm/venv/src/experiment-buddy clone Running command git fetch -q --tags Running command git reset --hard -q 8152622bea8f8dd5632797698ac564a3f6b0c30e Requirement already satisfied: gym in /home/mila/d/delvermm/venv/src/gym (from -r requirements.txt (line 2)) (0.15.6) Requirement already satisfied: matplotlib in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (3.4.1) Requirement already satisfied: numpy in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 4)) (1.20.2) Requirement already satisfied: pandas in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 5)) (1.2.3) Requirement already satisfied: pip in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 6)) (21.1.1) Requirement already satisfied: python-binance==0.7.9 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 7)) (0.7.9) Requirement already satisfied: scipy in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 8)) (1.6.2) Requirement already satisfied: setuptools in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 9)) (54.0.0) Requirement already satisfied: sru in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 10)) (2.5.1) Requirement already satisfied: tensorboard in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 11)) (2.4.1) Requirement already satisfied: tensorboardX in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 12)) (2.2) Requirement already satisfied: torch in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 13)) (1.7.0+cu92) Requirement already satisfied: tqdm in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 14)) (4.59.0) Requirement already satisfied: wandb in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 15)) (0.10.24) Requirement already satisfied: xarray==0.17.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 16)) (0.17.0) Requirement already satisfied: flake8 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 17)) (3.9.0) Requirement already satisfied: alphabet==0.0.8 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 18)) (0.0.8) Requirement already satisfied: tkintertable in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from -r requirements.txt (line 19)) (1.3.2) Requirement already satisfied: GitPython in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy->-r requirements.txt (line 1)) (3.1.14) Requirement already satisfied: fabric in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy->-r requirements.txt (line 1)) (2.6.0) Requirement already satisfied: cloudpickle in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy->-r requirements.txt (line 1)) (1.2.2) Requirement already satisfied: PyYaml in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy->-r requirements.txt (line 1)) (5.4.1) Requirement already satisfied: paramiko in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from experiment_buddy->-r requirements.txt (line 1)) (2.7.2) Requirement already satisfied: service-identity in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (18.1.0) Requirement already satisfied: Twisted in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (21.2.0) Requirement already satisfied: cryptography in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (3.4.7) Requirement already satisfied: urllib3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (1.26.4) Requirement already satisfied: autobahn in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (21.3.1) Requirement already satisfied: pyOpenSSL in /cvmfs/ai.mila.quebec/apps/x86_64/debian/python/3.7/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (19.1.0) Requirement already satisfied: six in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (1.15.0) Requirement already satisfied: dateparser in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (1.0.0) Requirement already satisfied: ujson in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (4.0.2) Requirement already satisfied: requests in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (2.25.1) Requirement already satisfied: certifi in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from python-binance==0.7.9->-r requirements.txt (line 7)) (2020.12.5) Requirement already satisfied: pygments in /cvmfs/ai.mila.quebec/apps/x86_64/debian/python/3.7/lib/python3.7/site-packages (from alphabet==0.0.8->-r requirements.txt (line 18)) (2.7.1) Requirement already satisfied: googletrans in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from alphabet==0.0.8->-r requirements.txt (line 18)) (3.0.0) Requirement already satisfied: filemagic in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from alphabet==0.0.8->-r requirements.txt (line 18)) (1.6) Requirement already satisfied: pytz>=2017.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from pandas->-r requirements.txt (line 5)) (2021.1) Requirement already satisfied: python-dateutil>=2.7.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from pandas->-r requirements.txt (line 5)) (2.8.1) Requirement already satisfied: pyglet<=1.5.0,>=1.4.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from gym->-r requirements.txt (line 2)) (1.5.0) Requirement already satisfied: future in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from pyglet<=1.5.0,>=1.4.0->gym->-r requirements.txt (line 2)) (0.18.2) Requirement already satisfied: pillow>=6.2.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->-r requirements.txt (line 3)) (7.2.0) Requirement already satisfied: cycler>=0.10 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->-r requirements.txt (line 3)) (0.10.0) Requirement already satisfied: pyparsing>=2.2.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->-r requirements.txt (line 3)) (2.4.7) Requirement already satisfied: kiwisolver>=1.0.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from matplotlib->-r requirements.txt (line 3)) (1.3.1) Requirement already satisfied: ninja in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from sru->-r requirements.txt (line 10)) (1.10.0.post2) Requirement already satisfied: dataclasses in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from torch->-r requirements.txt (line 13)) (0.6) Requirement already satisfied: typing-extensions in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from torch->-r requirements.txt (line 13)) (3.7.4.3) Requirement already satisfied: werkzeug>=0.11.15 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (1.0.1) Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (1.8.0) Requirement already satisfied: protobuf>=3.6.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (3.15.7) Requirement already satisfied: grpcio>=1.24.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (1.37.0) Requirement already satisfied: absl-py>=0.4 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (0.12.0) Requirement already satisfied: wheel>=0.26 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (0.36.2) Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (0.4.4) Requirement already satisfied: markdown>=2.6.8 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (3.3.4) Requirement already satisfied: google-auth<2,>=1.6.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from tensorboard->-r requirements.txt (line 11)) (1.28.0) Requirement already satisfied: cachetools<5.0,>=2.0.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard->-r requirements.txt (line 11)) (4.2.1) Requirement already satisfied: rsa<5,>=3.1.4 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard->-r requirements.txt (line 11)) (4.7.2) Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard->-r requirements.txt (line 11)) (0.2.8) Requirement already satisfied: requests-oauthlib>=0.7.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->-r requirements.txt (line 11)) (1.3.0) Requirement already satisfied: importlib-metadata in /cvmfs/ai.mila.quebec/apps/x86_64/debian/python/3.7/lib/python3.7/site-packages (from markdown>=2.6.8->tensorboard->-r requirements.txt (line 11)) (1.5.0) Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard->-r requirements.txt (line 11)) (0.4.8) Requirement already satisfied: chardet<5,>=3.0.2 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from requests->python-binance==0.7.9->-r requirements.txt (line 7)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from requests->python-binance==0.7.9->-r requirements.txt (line 7)) (2.10) Requirement already satisfied: oauthlib>=3.0.0 in /cvmfs/ai.mila.quebec/apps/x86_64/debian/python/3.7/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->-r requirements.txt (line 11)) (3.1.0) Requirement already satisfied: configparser>=3.8.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (5.0.2) Requirement already satisfied: shortuuid>=0.5.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (1.0.1) Requirement already satisfied: Click>=7.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (7.1.2) Requirement already satisfied: docker-pycreds>=0.4.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (0.4.0) Requirement already satisfied: promise<3,>=2.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (2.3) Requirement already satisfied: psutil>=5.0.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (5.8.0) Requirement already satisfied: pathtools in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (0.1.2) Requirement already satisfied: subprocess32>=3.5.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (3.5.4) Requirement already satisfied: sentry-sdk>=0.4.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from wandb->-r requirements.txt (line 15)) (1.0.0) Requirement already satisfied: gitdb<5,>=4.0.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from GitPython->experiment_buddy->-r requirements.txt (line 1)) (4.0.7) Requirement already satisfied: smmap<5,>=3.0.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from gitdb<5,>=4.0.1->GitPython->experiment_buddy->-r requirements.txt (line 1)) (4.0.0) Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from flake8->-r requirements.txt (line 17)) (0.6.1) Requirement already satisfied: pycodestyle<2.8.0,>=2.7.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from flake8->-r requirements.txt (line 17)) (2.7.0) Requirement already satisfied: pyflakes<2.4.0,>=2.3.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from flake8->-r requirements.txt (line 17)) (2.3.1) Requirement already satisfied: hyperlink>=21.0.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from autobahn->python-binance==0.7.9->-r requirements.txt (line 7)) (21.0.0) Requirement already satisfied: txaio>=21.2.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from autobahn->python-binance==0.7.9->-r requirements.txt (line 7)) (21.2.1) Requirement already satisfied: cffi>=1.12 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from cryptography->python-binance==0.7.9->-r requirements.txt (line 7)) (1.14.5) Requirement already satisfied: pycparser in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from cffi>=1.12->cryptography->python-binance==0.7.9->-r requirements.txt (line 7)) (2.20) Requirement already satisfied: regex!=2019.02.19 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from dateparser->python-binance==0.7.9->-r requirements.txt (line 7)) (2021.4.4) Requirement already satisfied: tzlocal in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from dateparser->python-binance==0.7.9->-r requirements.txt (line 7)) (2.1) Requirement already satisfied: invoke<2.0,>=1.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from fabric->experiment_buddy->-r requirements.txt (line 1)) (1.5.0) Requirement already satisfied: pathlib2 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from fabric->experiment_buddy->-r requirements.txt (line 1)) (2.3.5) Requirement already satisfied: bcrypt>=3.1.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from paramiko->experiment_buddy->-r requirements.txt (line 1)) (3.2.0) Requirement already satisfied: pynacl>=1.0.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from paramiko->experiment_buddy->-r requirements.txt (line 1)) (1.4.0) Requirement already satisfied: httpx==0.13.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (0.13.3) Requirement already satisfied: sniffio in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from httpx==0.13.3->googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (1.2.0) Requirement already satisfied: hstspreload in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from httpx==0.13.3->googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (2020.12.22) Requirement already satisfied: httpcore==0.9.* in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from httpx==0.13.3->googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (0.9.1) Requirement already satisfied: rfc3986<2,>=1.3 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from httpx==0.13.3->googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (1.4.0) Requirement already satisfied: h11<0.10,>=0.8 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from httpcore==0.9.*->httpx==0.13.3->googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (0.9.0) Requirement already satisfied: h2==3.* in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from httpcore==0.9.*->httpx==0.13.3->googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (3.2.0) Requirement already satisfied: hyperframe<6,>=5.2.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from h2==3.*->httpcore==0.9.*->httpx==0.13.3->googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (5.2.0) Requirement already satisfied: hpack<4,>=3.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from h2==3.*->httpcore==0.9.*->httpx==0.13.3->googletrans->alphabet==0.0.8->-r requirements.txt (line 18)) (3.0.0) Requirement already satisfied: zipp>=0.5 in /cvmfs/ai.mila.quebec/apps/x86_64/debian/python/3.7/lib/python3.7/site-packages (from importlib-metadata->markdown>=2.6.8->tensorboard->-r requirements.txt (line 11)) (3.0.0) Requirement already satisfied: attrs>=16.0.0 in /cvmfs/ai.mila.quebec/apps/x86_64/debian/python/3.7/lib/python3.7/site-packages (from service-identity->python-binance==0.7.9->-r requirements.txt (line 7)) (20.2.0) Requirement already satisfied: incremental>=16.10.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from Twisted->python-binance==0.7.9->-r requirements.txt (line 7)) (21.3.0) Requirement already satisfied: zope.interface>=4.4.2 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from Twisted->python-binance==0.7.9->-r requirements.txt (line 7)) (5.3.0) Requirement already satisfied: Automat>=0.8.0 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from Twisted->python-binance==0.7.9->-r requirements.txt (line 7)) (20.2.0) Requirement already satisfied: constantly>=15.1 in /home/mila/d/delvermm/venv/lib/python3.7/site-packages (from Twisted->python-binance==0.7.9->-r requirements.txt (line 7)) (15.1.0) Installing collected packages: experiment-buddy Attempting uninstall: experiment-buddy Found existing installation: experiment-buddy 0.0.1 Uninstalling experiment-buddy-0.0.1: Successfully uninstalled experiment-buddy-0.0.1 Running setup.py develop for experiment-buddy Successfully installed experiment-buddy-0.0.1 [DEPLOY LOG] /opt/slurm/bin/sbatch /tmp/experiment_buddy-zCE8iXKnF4/srun_python.sh train.py Submitted batch job 939092 100%|██████████| 1/1 [02:11<00:00, 131.39s/it] Process finished with exit code 0