Too many values to unpack (expected 4) on LambdaLabs

[steventrouble]

Running on a new LambdaLabs instance (A100 x1, 40GB) returns an error in selfplay_worker.py:

ValueError: too many values to unpack (expected 4)

The issue seems likely to be a version mismatch between gym and some other library we didn't specify the version of in the requirements.txt. Long term, we'll want to update everything to use gymnasium and the latest versions of ALE, but for now we need to figure out which package is causing this conflict and freeze it in the requirements file.

Full error below.

Traceback (most recent call last):
File "/home/ubuntu/EfficientZero/core/selfplay_worker.py", line 107, in run
  self._run()
File "/home/ubuntu/.local/share/virtualenvs/EfficientZero-KqajkB4Z/lib/python3.8/site-packages/ray/util/tracing/tracing_helper.py", line 460, in _resume_span
  return method(self, *_args, **_kwargs)
File "/home/ubuntu/EfficientZero/core/selfplay_worker.py", line 139, in _run
  init_obses = [env.reset() for env in envs]
File "/home/ubuntu/EfficientZero/core/selfplay_worker.py", line 139, in <listcomp>
  init_obses = [env.reset() for env in envs]
File "/home/ubuntu/EfficientZero/config/atari/env_wrapper.py", line 37, in reset
  observation = self.env.reset(**kwargs)
File "/home/ubuntu/.local/share/virtualenvs/EfficientZero-KqajkB4Z/lib/python3.8/site-packages/gym/core.py", line 311, in reset
  return self.observation(self.env.reset(**kwargs))
File "/home/ubuntu/EfficientZero/core/utils.py", line 120, in reset
  obs = self.env.reset(**kwargs)
File "/home/ubuntu/EfficientZero/core/utils.py", line 58, in reset
  return self.env.reset(**kwargs)
File "/home/ubuntu/EfficientZero/core/utils.py", line 155, in reset
  return self.env.reset(**kwargs)
File "/home/ubuntu/EfficientZero/core/utils.py", line 82, in reset
  obs, _, done, _ = self.env.step(self.noop_action)
File "/home/ubuntu/.local/share/virtualenvs/EfficientZero-KqajkB4Z/lib/python3.8/site-packages/gym/wrappers/order_enforcing.py", line 13, in step
  observation, reward, done, info = self.env.step(action)
ValueError: too many values to unpack (expected 4)

[steventrouble]

On it. I ran pip freeze on the stability cluster and diffed it with LambdaLabs. Looks like there's quite a few differences there, so it won't be a super easy fix. I don't see any that look like they'd obviously cause an issue.

I'm assuming there's only one package that's mismatched, so I'll do a quick binary search on the packages to see which fixes the failure.