Update HER + update to gym 0.26+

[arjun-kg]

To be merged after next SB3 release. Notebooks run with SB3 at this fork (https://github.com/DLR-RM/stable-baselines3/pull/780)

1. Updated to new seed method in one file
2. Updated HER to be compatible with latest SB3

[tlpss]

The wandb integration is not working for gym v0.26 yet, see https://github.com/wandb/wandb/issues/4305. A PR seems to be in place so I'm hoping this gets merged and released soon.

[tlpss]

@araffin 3 notebooks are not working yet for gym v0.26:

• pybullet_envs -> ~no word on gymv0.26 yet~ Bullet3 in progress (https://github.com/bulletphysics/bullet3/pull/4335)
• wandb -> integration for gymv0.26 in progress (https://github.com/wandb/wandb/issues/4305)
• rl-zoo -> integration in progress (https://github.com/DLR-RM/rl-baselines3-zoo/pull/256)

[araffin]

pybullet_envs -> no word on gym v0.26 yet, repo seems to have low activity (pybullet is no longer actively developed after all)

Pull request is here: https://github.com/bulletphysics/bullet3/pull/4335

Pybullet is no longer actively maintained but that doesn't mean it's not usable/useful (especially pybullet envs are more interestings and challenging than the mujoco equivalents).

[tlpss]

Pull request is here: https://github.com/bulletphysics/bullet3/pull/4335

Ah my bad, I was looking in the pybullet-gym repo.

Pybullet is no longer actively maintained but that doesn't mean it's not usable/useful

Couldn't agree more! I have been using it extensively the past months.

[araffin]

pybullet_envs -> no word on gymv0.26 yet Bullet3 in progress

~we could activate the compatibility in the meantime, no?~

Nevermind, gym breaks pybullet by breaking changes in its registry...

[araffin]

pybullet_envs -> no word on gymv0.26 yet Bullet3 in progress (compatibility with Gym 0.26.0 bulletphysics/bullet3#4335)

We should probably patch it the way I did for the RL Zoo: https://github.com/bulletphysics/bullet3/issues/4368#issuecomment-1363269019

EDIT: done