Modify environments so they can be deepcopied

CarlosGual commented 2 years ago

First of all, thank you very much for your work and contribution to the reinforcement learning community! :cowboy_hat_face:

Problem

I am trying to use the Mazes environments into garage library. However, I've been strugling a lot until I found what I think it is the main issue. In garage, some algorithms implement a task sampler to create different instances of the environments. To achieve this, they do a deepcopy of the environments. Here is where I found that MiniWorld environments can't be deepcopied. I just would like to know how to modify them in order to be deepcopied.

Code Snippet

Here you are a code snippet to reproduce the issue:

import gym
import gym_miniworld
import copy

env = gym.make('MiniWorld-MazeS3Fast-v0')
copy.deepcopy(env)

Traceback

If you execute the previous code, you will see an error traceback like this one:

Traceback (most recent call last):
  File "/home/carlos/repositorios/garage/development/pruebas_envdeepcopy.py", line 6, in <module>
    copy.deepcopy(env)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 281, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 281, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 281, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 281, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 281, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 281, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 281, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/carlos/miniconda3/envs/garage/lib/python3.7/copy.py", line 169, in deepcopy
    rv = reductor(4)
ValueError: ctypes objects containing pointers cannot be pickled

Solution

For the moment I haven't been able to find any solution to this problem. It is the first time I use deepcopy, so I am not an expert. Also, I do not know enough of the MiniWorld internal workings, so I can't figure where to start out. I just would like to ask you to suggest me some hints in order to know where to start modifying the environments. Maybe just some kind of wrapper will make it work.

Thanks in advance! :smile_cat:

maximecb commented 2 years ago

This doesn't work because the numpy arrays don't support deepcopy, but I think that a custom deepcopy implementation is possible. I would be open to accepting a pull request if someone is up for implementing it.

CarlosGual commented 2 years ago

Hi, thank you for your comments. As you mentioned, I did a custom deepcopy implementation in the maze environment and now I am able to deepcopy it. I have to test that it works properly and that the results that I obtain with the env are consistent. If so, I will do myself the pull request.

jkterry1 commented 1 year ago

EZpickling should be supported instead

Farama-Foundation / Miniworld