In the LogWrapper initialization, both returned_episode_returns and returned_episode_lengths are set to 0.
Since 0 is a valid return for many environments, is it impossible (in isolation) to tell apart an initial return of 0 and a non-terminated first episode.
In order to indicate missing data before the first episode terminates, these should be set to NaN (or None).
In the
LogWrapperinitialization, bothreturned_episode_returnsandreturned_episode_lengthsare set to 0.Since 0 is a valid return for many environments, is it impossible (in isolation) to tell apart an initial return of 0 and a non-terminated first episode.
In order to indicate missing data before the first episode terminates, these should be set to
NaN(orNone).