search

vivekratnavel / omniboard

Web-based dashboard for Sacred
MIT License
548 stars 46 forks source link

Feature request: Config values in compare plot legend #216

Closed MaxSchambach closed 4 years ago

MaxSchambach commented 4 years ago

I have mentioned this in the closed issue #180 : While this one has been closed and it now possible to use the Experiment Name as the plot label, this does not fully work for many workflows (including mine, obviously).

I often use the Experiment Name to group single runs with a common experiment target. Then, when comparing many runs, the ID is not very informative (as mentioned in #180 ) and the Experiment Name is not necessarily unique.

What I would suggest is to make it possible to add arbitrary (possibly multiple, concatenated) configuration parameters to the label. This would make it very straightforward to compare different configuration hyperparameters, such as different model topologies, used optimizers, batch sizes (in case of ML) and so on. For me, this would be a huge improvement when evaluating experiments with Omniboard! (Otherwise, I am, too, really enjoying working with Omniboard, it's a super useful tool!)

Ideally, there would be some sort of 'plus buttom' to concatenate legend label strings with arbitrary configuration parameters. I have tried fiddling around with the corresponding JavaScript files, however my JavaScript skills are virtually non-existent, so I failed and unfortunately can't help with a pull request.

vivekratnavel commented 4 years ago
Screen Shot 2020-05-28 at 1 37 11 AM

You can now add multiple metric labels with a wide range of options to choose from the dropdown.

@MaxSchambach Let me know how you like the new feature. Thanks!

vivekratnavel commented 4 years ago

:tada: This issue has been resolved in version 2.11.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

MaxSchambach commented 4 years ago

Very cool, thanks! Just tried it out, pretty much exactly what I thought it could work like. Thank you for the super quick implementation!

vivekratnavel commented 4 years ago

@MaxSchambach Thanks for your UI design idea! It helped me to quickly implement the feature. Thanks for trying out quickly and I am glad it works for you!

alvarovalcarce commented 4 years ago

Are some type of configuration parameter excluded from the dropdown list? I have some configuration parameters in the form of a list that don't show up here.

vivekratnavel commented 4 years ago

@alvarovalcarce Yes, only strings are shown. If it is a dictionary, it will get expanded.