Support gym.spaces.Dict

[iory]

There is some gym environment using gym.spaces.Dict. For example, hand manipulation environment's observation_space has three property, desired_goal, achieved_goal, and observation. https://github.com/openai/gym/blob/master/gym/envs/robotics/hand/manipulate.py#L262-L271 https://github.com/openai/gym/blob/master/gym/envs/robotics/hand/manipulate_touch_sensors.py#L60-L64

It seems that in current epi_sampler implementation gym.spaces.Dict not supported.

[takerfume]

219

[takerfume]

All that is left is to support a nested dictionary of gym.spaces.Dict like the following example. This is written in gym.spaces.dict_space.py

Example usage [nested]:
self.nested_observation_space = spaces.Dict({
    'sensors':  spaces.Dict({
        'position': spaces.Box(low=-100, high=100, shape=(3)),
        'velocity': spaces.Box(low=-1, high=1, shape=(3)),
        'front_cam': spaces.Tuple((
            spaces.Box(low=0, high=1, shape=(10, 10, 3)),
            spaces.Box(low=0, high=1, shape=(10, 10, 3))
        )),
        'rear_cam': spaces.Box(low=0, high=1, shape=(10, 10, 3)),
    }),
    'ext_controller': spaces.MultiDiscrete([ [0,4], [0,1], [0,1] ]),
    'inner_state':spaces.Dict({
        'charge': spaces.Discrete(100),
        'system_checks': spaces.MultiBinary(10),
        'job_status': spaces.Dict({
            'task': spaces.Discrete(5),
            'progress': spaces.Box(low=0, high=100, shape=()),
        })
    })
})