Basic dict obs support

[araffin]

Description

• Add Gymnasium support, via https://github.com/DLR-RM/stable-baselines3/pull/1327
• Flexible mlp for off-policy algorithms
• basic dict support / HerReplayBuffer

Motivation and Context

• [ ] I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] Documentation (update in the documentation)

Checklist:

• [ ] I've read the CONTRIBUTION guide (required)
• [ ] I have updated the changelog accordingly (required).
• [ ] My change requires a change to the documentation.
• [ ] I have updated the tests accordingly (required for a bug fix or a new feature).
• [ ] I have updated the documentation accordingly.
• [ ] I have reformatted the code using make format (required)
• [ ] I have checked the codestyle using make check-codestyle and make lint (required)
• [ ] I have ensured make pytest and make type both pass. (required)
• [ ] I have checked that the documentation builds using make doc (required)

Note: You can run most of the checks using make commit-checks.

Note: we are using a maximum length of 127 characters per line