[Bug Report] Env Checker (reset with seed): This should never happen

[axelbr]

Describe the bug

When checking my environment, the check_reset_seed test fails and I get the following error: This should never happen, please report this issue. The error was: cannot pickle 'generator' object

It looks like it is related to line 85 in order_enforcing.py. When deepcopying the env_spec (which holds a Generator object), it fails.

Code example

No response

System info

• pip
• gymnasium==0.28.1
• Ubuntu 22.04
• 3.8.17

Additional context

No response

Checklist

• [X] I have checked that there is no similar issue in the repo

[pseudo-rnd-thoughts]

Interesting bug, could you provide example code to replicate the issue?

[axelbr]

Well, i just call the check on my custom env: env_checker.check_env(env, skip_render_check=True) My environment is not publicly available yet, but i do not set or touch this generator.

[pseudo-rnd-thoughts]

Could you produce an example script for me to test with? I suspect that one of your environment parameters is a generator not a tuple

[axelbr]

Ok. I figured out that the deepcopy fails, because my own code provides some non-pickleable objects. Here's an example:

from __future__ import annotations
from typing import Any
import gymnasium
from gymnasium.core import ObsType
from gymnasium.utils import env_checker

class Unpickleable:
    def __getstate__(self):
        raise RuntimeError('Cannot pickle me!')

class CustomEnv(gymnasium.Env):

    def __init__(self, unpickleable) -> None:
        self.action_space = gymnasium.spaces.Discrete(2)
        self.observation_space = gymnasium.spaces.Discrete(2)
        super().__init__()

    def step(self, action):
        return self.observation_space.sample(), 0, False, False, {}

    def reset(self, *, seed: int | None = None, options: dict[str, Any] | None = None) -> tuple[
        ObsType, dict[str, Any]]:
        super().reset(seed=seed, options=options)
        if seed is not None:
            self.observation_space.seed(seed)
        return self.observation_space.sample(), {}

gymnasium.register(
    id='Custom-v0',
    entry_point='__main__:CustomEnv',
    kwargs={},
)

if __name__ == '__main__':
    env = gymnasium.make('Custom-v0', unpickleable=Unpickleable())
    env_checker.check_env(env, skip_render_check=True)

[axelbr]

I updated the script to provide the unpickleable object at env creation. This reproduces exactly my error.

[RedTachyon]

The problem seems to happen specifically when the argument in an env's constructor's arguments cannot be copied. Some logic in the wrappers (including the env checker) creates a deepcopy of the env spec so that the wrapper's spec can be appropriately modified in the wrappers, without modifying the underlying env's spec.

It'd definitely be good to improve the error message, but I'm not sure if there's anything else we can/should do. Can you describe what use-case you have where you're passing uncopiable/unpicklable objects to the constructor? (I'm trying to assess if this is a very unusual edge case, or something we need to seriously consider)

[axelbr]

My specific use case is that I pass CARLA objects (e.g. a world instance). However, there are many use cases where we deal with non-pickleable objects, such as network connections.

[pseudo-rnd-thoughts]

Fixed in #982