Not entirely sure whats going wrong??

[shadowbourne]

PS C:\Users\shadow\Desktop\tb> tb-reducer -i 'seed?/*' -o out Loaded 2 TensorBoard runs with 99 scalars and 383 steps each

• eval-SuperMarioBros-1-1-v1
• eval-lives-SuperMarioBros-1-1-v1
• eval-x-SuperMarioBros-1-1-v1
• eval-SuperMarioBros-2-1-v1
• eval-lives-SuperMarioBros-2-1-v1
• eval-x-SuperMarioBros-2-1-v1
• eval-SuperMarioBros-3-2-v1
• eval-lives-SuperMarioBros-3-2-v1
• eval-x-SuperMarioBros-3-2-v1
• eval-SuperMarioBros-6-4-v1
• eval-lives-SuperMarioBros-6-4-v1
• eval-x-SuperMarioBros-6-4-v1
• new episodes per training epoch
• new sequences per training epoch
• multienvironment efficiency
• self.score
• self.eval_score
• self.x
• self.finishing_rate
• self.loss
• eps
• eval-SuperMarioBros-1-2-v1
• eval-lives-SuperMarioBros-1-2-v1
• eval-x-SuperMarioBros-1-2-v1
• eval-SuperMarioBros-1-3-v1
• eval-lives-SuperMarioBros-1-3-v1
• eval-x-SuperMarioBros-1-3-v1
• eval-SuperMarioBros-1-4-v1
• eval-lives-SuperMarioBros-1-4-v1
• eval-x-SuperMarioBros-1-4-v1
• eval-SuperMarioBros-2-2-v1
• eval-lives-SuperMarioBros-2-2-v1
• eval-x-SuperMarioBros-2-2-v1
• eval-SuperMarioBros-2-3-v1
• eval-lives-SuperMarioBros-2-3-v1
• eval-x-SuperMarioBros-2-3-v1
• eval-SuperMarioBros-2-4-v1
• eval-lives-SuperMarioBros-2-4-v1
• eval-x-SuperMarioBros-2-4-v1
• eval-SuperMarioBros-3-1-v1
• eval-lives-SuperMarioBros-3-1-v1
• eval-x-SuperMarioBros-3-1-v1
• eval-SuperMarioBros-3-3-v1
• eval-lives-SuperMarioBros-3-3-v1
• eval-x-SuperMarioBros-3-3-v1
• eval-SuperMarioBros-3-4-v1
• eval-lives-SuperMarioBros-3-4-v1
• eval-x-SuperMarioBros-3-4-v1
• eval-SuperMarioBros-4-1-v1
• eval-lives-SuperMarioBros-4-1-v1
• eval-x-SuperMarioBros-4-1-v1
• eval-SuperMarioBros-4-2-v1
• eval-lives-SuperMarioBros-4-2-v1
• eval-x-SuperMarioBros-4-2-v1
• eval-SuperMarioBros-4-3-v1
• eval-lives-SuperMarioBros-4-3-v1
• eval-x-SuperMarioBros-4-3-v1
• eval-SuperMarioBros-4-4-v1
• eval-lives-SuperMarioBros-4-4-v1
• eval-x-SuperMarioBros-4-4-v1
• eval-SuperMarioBros-5-1-v1
• eval-lives-SuperMarioBros-5-1-v1
• eval-x-SuperMarioBros-5-1-v1
• eval-SuperMarioBros-5-2-v1
• eval-lives-SuperMarioBros-5-2-v1
• eval-x-SuperMarioBros-5-2-v1
• eval-SuperMarioBros-5-3-v1
• eval-lives-SuperMarioBros-5-3-v1
• eval-x-SuperMarioBros-5-3-v1
• eval-SuperMarioBros-5-4-v1
• eval-lives-SuperMarioBros-5-4-v1
• eval-x-SuperMarioBros-5-4-v1
• eval-SuperMarioBros-6-1-v1
• eval-lives-SuperMarioBros-6-1-v1
• eval-x-SuperMarioBros-6-1-v1
• eval-SuperMarioBros-6-2-v1
• eval-lives-SuperMarioBros-6-2-v1
• eval-x-SuperMarioBros-6-2-v1
• eval-SuperMarioBros-6-3-v1
• eval-lives-SuperMarioBros-6-3-v1
• eval-x-SuperMarioBros-6-3-v1
• eval-SuperMarioBros-7-1-v1
• eval-lives-SuperMarioBros-7-1-v1
• eval-x-SuperMarioBros-7-1-v1
• eval-SuperMarioBros-7-2-v1
• eval-lives-SuperMarioBros-7-2-v1
• eval-x-SuperMarioBros-7-2-v1
• eval-SuperMarioBros-7-3-v1
• eval-lives-SuperMarioBros-7-3-v1
• eval-x-SuperMarioBros-7-3-v1
• eval-SuperMarioBros-8-1-v1
• eval-lives-SuperMarioBros-8-1-v1
• eval-x-SuperMarioBros-8-1-v1
• eval-SuperMarioBros-8-2-v1
• eval-lives-SuperMarioBros-8-2-v1
• eval-x-SuperMarioBros-8-2-v1
• eval-SuperMarioBros-8-3-v1
• eval-lives-SuperMarioBros-8-3-v1
• eval-x-SuperMarioBros-8-3-v1 Writing 'mean' reduction to 'out-mean' PS C:\Users\shadow\Desktop\tb>

[shadowbourne]

But please help as I think this tool is (or would be if i could get it to work) so useful!

[janosh]

I haven't tested tb-reducer on Windows so maybe a path separator issue? Could you zip and upload your event files here?

[shadowbourne]

files.zip

[janosh]

Thanks for your files. So there were multiple problems here.

First, tb-reducer rather short-sightedly did not yet support logs increasing in anything other than unit steps. It had to be

[
    {"step": 0, "value": 0.8129},
    {"step": 1, "value": 7234},
    {"step": 2, "value": 0.5224},
    ...
]

This is now fixed in tensorboard-reducer==0.2.0.

Second, it intentionally did not support runs with different step counts. In fact, there was a disclaimer in the readme about that (now removed). I should have looked at your screenshot more carefully. It clearly shows that seed0 reaches to about 50k steps, while seed1 only has ~18k steps. You can now explicitly tell tb-reducer that you are aware of this and would like to reduce those two runs anyway (using however many steps are available in the shortest run) by passing the --lax-steps flag.

Third, TensorBoard Reducer also intentionally did not support loading TensorBoard runs with non-identical sets of scalar tags. This constraint can also now be deactivated by passing --lax-tags. However, this wasn't an issue in your case.

In summary, could you try pip install -U tensorboard-reducer to make sure you're running 0.2.0 followed by tb-reducer -i 'seed?/*' -o out --lax-steps and see if it works now?

[shadowbourne]

Lovely work! new issues have emerged, but I shall close this issue and open another!