The normalization wrapper in Sinergym, based on the default Gymnasium wrapper, has encountered a functionality issue.
When a model is trained using this normalization, the calibration is lost post-process. Consequently, when the model is loaded for evaluation, the absence of normalization calibration results in poor agent performance.
The wrapper should be capable of recording the averages and deviations updated during this process, and offer the ability to load them either in the constructor or via a method.
Furthermore, it may not always be desirable to continue updating calibrations by default during the evaluation process. Hence, a feature to disable or enable these automatic updates during environment interaction is proposed.
Additional context
Example notebooks illustrating the use and utility of this new feature should be included.
Checklist
[x] I have checked that there is no similar issue in the repo (required)
:pencil: Please, don't forget to include more labels besides Feature request if it is necessary.
Feature 🚀
The normalization wrapper in Sinergym, based on the default Gymnasium wrapper, has encountered a functionality issue.
When a model is trained using this normalization, the calibration is lost post-process. Consequently, when the model is loaded for evaluation, the absence of normalization calibration results in poor agent performance.
The wrapper should be capable of recording the averages and deviations updated during this process, and offer the ability to load them either in the constructor or via a method.
Furthermore, it may not always be desirable to continue updating calibrations by default during the evaluation process. Hence, a feature to disable or enable these automatic updates during environment interaction is proposed.
Additional context
Example notebooks illustrating the use and utility of this new feature should be included.
Checklist
:pencil: Please, don't forget to include more labels besides
Feature requestif it is necessary.