Add support for the new OpenAI Gym `step` API interface

dstl / YAWNING-TITAN

YAWNING TITAN is an abstract, graph based cyber-security simulation environment that supports the training of intelligent agents for autonomous cyber operations.

MIT License

58 stars 17 forks source link

Add support for the new OpenAI Gym `step` API interface #4

Open br0kej opened 2 years ago

br0kej commented 2 years ago

The upstream team developing gym have implemented a new step API interface to support more granular termination criteria for environments. The details of the changes can be found here - https://github.com/openai/gym/pull/2752.

This doesn't seem to be impacting the current build but as gym matures, it's likely that this interface will become the default.

jamesshort1 commented 2 years ago

Should be covered off when we implement the new "Gymnasium" interface. Will assess

ChrisMcCarthyDev commented 1 year ago

Just a quick update from our end. Stable Baselines3 and Gymnasium are now fully aligned in terms of compatible dependency versions, however Ray RLlib is still out of sync with the version of Gymnasium it depends on. This means we cannot migrate from Gym to Gymnasium just yet while still supporting Ray RLlib. We're hoping this issue is resolved soon so we can complete the migration, which will then bring in the new step API.

ChrisMcCarthyDev commented 1 year ago

This a quick update from the Yawning-Titan Dev Team to say that Ray RLlib(versions >= 2.7.0) now supports Gymnasium v0.28.1. With both Stable Baselines3 and Ray RLlib now aligned on the same version of Gymnasium, we can move forward with this ticket. It is on our backlog and will be progressed soon.