Truncation API Update

[jjshoots]

About

As part of the Gym update to 0.25, the following changes have been made:

done -> termination and truncation

The singular done signal has been changed to a termination and truncation signal, where termination dictates that the environment has ended due to meeting certain conditions, and truncation dictates that the environment has ended due to exceeding the time/frame limit

Progress

The road plan for this update is as follows:

1. Truncation/Termination update
   • [ ] AEC Vector
   • [ ] Generic Wrappers
   • [x] Lambda Wrappers
   • [ ] Multiagent Wrappers
   • [ ] Utils
   • [ ] Vector
2. Tests update
   • [x] vec_env_test.py
   • [x] test_autodep.py
   • [ ] pettingzoo_api_test.py
   • [x] parallel_env_test.py
   • [ ] gym_unwrapped_test.py
   • [ ] gym_mock_test.py
   • [ ] generated_agents_test.py
   • [x] aec_unwrapped_test.py
   • [x] aec_mock_test.py
   • [ ] test_vector/
   • [ ] test_utils/

[WillDudley]

Working on AECVector wrappers

[WillDudley]

Are you sure the aec_mock_test is done with? Seems like env.last() expects too few values. @jjshoots

I need a break after https://github.com/Farama-Foundation/SuperSuit/pull/182. But I can take a look at the test(s) later. But maybe I should move onto other wrappers just to get more wrappers minimally functional.

[jjshoots]

@WillDudley you're right, IDT that's done, my mistake.

[jjshoots]

@WillDudley I just merged #182 before realizing that the sanity checks code was included.

[WillDudley]

@jjshoots Was thinking it would be ok to keep them in until the actual tests were refactored, but if you want to remove them I don't mind

[WillDudley]

As Reggie is assigned to generic, I'll look at multiagent now.

[jjshoots]

@WillDudley also note that SS likely won't pass without the truncation update on PZ's master branch.

[WillDudley]

I'm using the API_Update PZ branch to develop SS, not the master branch.

https://github.com/Farama-Foundation/SuperSuit/pull/183 is done to a similar standard to the AECVector wrapper update - a minimal example runs.

Now that all 3 wrappers are hacked up (not including generic wrappers), I'll start looking at getting things to pass the actual tests.

[WillDudley]

I'm going to take a break. The current thing I'm stuck on is:

When calling pettingzoo/test/api_test.py, in line 268 where env.step(action) is called, action can be None, which causes an error down the thread at supersuit/utils/action_transforms/homogenize_ops.py, line 88 (evaluates to None > n - 1).

So I need to figure out where in the thread/stack is best to apply a fix for this, which either handles or catches cases where action=None. Options are shown attached

[benblack769]

@WillDudley Perhaps a simple change to the dehomogenize_actions function?

        if action is None:
             return None
        if action > n - 1:
            action = 0
        return action

[WillDudley]

Working on utils @ https://github.com/WillDudley/SuperSuit/pull/1