[Question] How to customize the loss calculation for PPO

[olmoulin]

❓ Question

Hi, For a research purpose, I need to modify the loss calculation for PPO in order to take into account another parameter ... I have modified the PPO.py file as follow : loss = (policy_loss + self.ent_coef entropy_loss + self.vf_coef value_loss + gen_loss), gen_loss being my new parameter ... but this has absolutely no impact on the training ... not sure, but am I making something wrong there ? thanks a lot in adance.

Checklist

• [X] I have checked that there is no similar issue in the repo
• [X] I have read the documentation
• [X] If code there is, it is minimal and working
• [X] If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[araffin]

Probably a duplicate of https://github.com/DLR-RM/stable-baselines3/issues/1881

[olmoulin]

This is what I have applied, and it does run properly my code, but it seems that no matter how much you modify the loss calculation it has no impact ... am I missing something ? Can you please then remove the duplicate tag on it. thanks