[Question] Support for Customized BaseFeaturesExtractor

[kingjin94]

❓ Question

I have implemented a FeatureExtractor as suggested in https://stable-baselines3.readthedocs.io/en/master/guide/custom_policy.html#custom-feature-extractor that I can manually train within SB3. It seems that using such a feature extractor has not yet been considered within the rl-baselines3-zoo. Is anyone aware of such efforts?

A preliminary survey of this code base indicates to me that an integration should be rather straight forward. It would require

1. some kind of registration such that the code of the custom Feature Extractor can be found and named (similar to ALGOS in utils.py or the registrations done in import_envs.py)
2. a slight change to ExperimentManager such that it adds "policy_kwargs" -> "features_extractor_class" based on the str to class mapping implemented in step 1.

I welcome any feedback on where else model creation is touched or where such a registration step for custom feature extractors would be appropriate.

Checklist

• [X] I have checked that there is no similar issue in the repo
• [X] I have read the SB3 documentation
• [X] I have read the RL Zoo documentation
• [X] If code there is, it is minimal and working
• [X] If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[araffin]

Duplicate of https://github.com/DLR-RM/rl-baselines3-zoo/issues/300, see the two solutions.