search

tensorflow / tensorboard

TensorFlow's Visualization Toolkit
Apache License 2.0
6.7k stars 1.66k forks source link

Tag experiments to filter and search according to hyper-parameters values #5372

Open aloisDeLaComble opened 3 years ago

aloisDeLaComble commented 3 years ago

As a data scientist, I am often running a lot of different experiments to compare different architecture designs ou hyper-parameter settings which I monitor in tensorboard. In order to differentiate and filter experiments in tensorboard, I'm giving long explicit names to my experiments such as "alois_text_and_image_lr_0001_bs_64_3_epochs..." but I cannot detail all the choices here and it's very painful to do. What I would like to do is to tag an experiment with a config object in my code (can be a dictionnary or a YACS config) and be able to filter and search experiments in tensorboard according to some parameters values (instead of the textual search).

If this is not already implemented, I can find some time to help on the implementation.

bileschi commented 3 years ago

This is a good idea. Users are often encoding hparam values into the run name, and the affordances we've built to select and color are name-based. Allowing a dictionary here with access to hparam values would be helpful

nfelt commented 3 years ago

Seems related to the longstanding #194 and #210.

pindinagesh commented 3 years ago

@aloisDeLaComble

Could you please respond on the above @nfelt comment. Thanks

nfelt commented 3 years ago

There's no need to for the OP to specifically respond to my comment, it was just an observation. I think this feature request is sufficiently distinct because it's specifically about hyperparameters (rather than generic tagging), but implementation-wise it could end up being similar and related.

aloisDeLaComble commented 3 years ago

yes indeed it's close to #210 but maybe simpler. In #210, tags can be set after running the exp. Here the idea is to set them once and for all when creating the experiment. It's also close to #194 but I'm not sure I got this one, the thread seems to be talking about different ideas but the request is about having a general metadata that would describe a set of experiments.