instadeepai / Mava

🦁 A research-friendly codebase for fast experimentation of multi-agent reinforcement learning in JAX
Apache License 2.0
737 stars 90 forks source link

[BUG] steps_per_second under-reporting in SAC and IQL #1078

Open JemmaLDaniel opened 6 months ago

JemmaLDaniel commented 6 months ago

Describe the bug

The (originally combined) steps_per_second logging has been separated for now to enforce logging consistencies across Mava systems, enabling downstream aggregate plotting via marl-eval. However, the current steps_per_second logging in the IQL and SAC systems is not a true representation for the learner or the actor. It is not entirely fair to say that we do steps_per_rollout / elapsed _time steps per second in both cases because we are also taking anakin_steps * cfg.system.epochs gradient steps.