search

interpreting-rl-behavior / interpreting-rl-behavior.github.io

Code for the site https://interpreting-rl-behavior.github.io/
Creative Commons Attribution 4.0 International
0 stars 0 forks source link

Fix losses. There is something fishing going on. #53

Closed leesharkey closed 2 years ago

leesharkey commented 2 years ago

loss_over_time_plot

One fishy thing is that the loss appears to get better over time. There's no reason to believe that that's an expected behaviour.

The main fishy thing is that the kl_rssm loss sometimes is negative. This shouldn't ever happen. It should be approximately the same size as the reconstruction loss (same OOM anyway).

leesharkey commented 2 years ago

Actually, the loss might get better over time because episodes end and thus the image goes all black, which is easy to predict.

The negative KL rssm loss is still a major red flag though.

leesharkey commented 2 years ago

fixed in cd7ec49e364cc269f11fdda2a68f2e1843839fb3 and before

still havent fixed the episodes ending issue but that's much less important