icaros-usc / pyribs

A bare-bones Python library for quality diversity optimization.
https://pyribs.org
MIT License
208 stars 32 forks source link

Update lunar lander to use Gymnasium #289

Closed btjanaka closed 1 year ago

btjanaka commented 1 year ago

Description

Summary: Gymnasium upgrade -> Tutorial and example modifications -> New lunar lander videos due to change in environment seed

OpenAI Gym has been succeeded by Gymnasium from the Farama foundation. Gymnasium has slowly been gaining adoption in RL projects. For instance, Ray/RLlib now uses it, though stablebaselines3 does not. This PR updates the lunar lander example and tutorial to use Gymnasium.

This update does interfere with the environment layout as we depend on a single seed for the environment layout and the random number generator changed in Gymnasium, but we found a new seed that gives a similar environment.

Note: Using %pip install moviepy>=1.0.0 doesn't seem to work on Colab as the > sign probably gets parsed weirdly. However, %pip install "moviepy>=1.0.0" does work.

Note: These are the weights for the policies in the videos. The environment seed is 52.

vertical:
[ 1.1734602, -4.335041,   3.6837032,  3.6420521,  2.1022677,  1.31881358, 0.8692830,  2.8115226,  1.6622023, -1.6422047,  1.8511474, -2.08029947, -6.7142382, -3.2570904, -0.114118,   0.8087383, -3.3746461, -1.60995059, -0.2695452, -5.4448816,  0.1738725, -4.5490489, -1.2642821, -1.34785392, -2.5572075,  3.0634394, -1.5778290, -0.483712,   2.8454288, -0.18931501, -0.6401813,  0.36698444]
left:
[2.35444723, 2.10489016, -0.06412905, 1.25785575, -0.61083637, -0.96629096, -2.71046417, -4.04866138, 1.30817786, 2.08277684, -0.88153194, -0.7520072, -4.51991445, -6.85092993, -0.5565642, 1.3714289, -0.06130933, -0.58282259, -4.59462963, -5.76532862, 1.34833532, 1.501134, 0.86583201, -1.54512854, -1.44776826, 1.25557653, -3.00444532, 3.38553445, 4.8837938, 0.74421563, 2.21713123, 2.30708639]
right:
[-4.2715897809929855, 0.9094442874581388, -5.202703117782204, 10.982891694777631, 5.605215839788631, -2.1405452251755386, 4.1121572222825895, -2.990895991105634, 0.5083944420485865, -11.757635520380054, 1.5665903927068037, 4.445223911903734, -7.881363189349994, 1.373437697702743, 2.7350996288563687, -7.744019587935334, -6.062981925884731, -2.5328208705039073, -9.864984842709475, -21.096489355976804, -1.9995229363964124, -3.5760704728662907, -2.8508148334162824, -0.14760450605032757, 4.67075828430291, 4.148793745645236, -17.775822302777353, 0.5349655001829168, 2.535914623989465, 9.302071658478983, -14.716597223781651, -1.651919392150396]

TODO

Questions

Status