Hierarchical display for TensorBoard

alexcoca commented 4 years ago

System information

TensorFlow version (you are using): 2.1
Are you willing to contribute it (Yes/No): Not a frontend guy so I don't know if I can help.

Describe the feature and the current behavior/state. I'm using TensorBoard to display an optimisation process with two stages, lambda_sweep and counterfactual_search, so I used the name argument to the summary ops to display the plots for the two stages separtely:

I would like this behaviour to be hierarchical, that is, other drop-downs would appear when clicking one of the stages. Instead, I get the follwing:

I was wondering if there is any way to extend the first level logic so that, for example, under counterfactual_search I get another page I can click on that is called lambda and so on.

Will this change the current api? How? I don't think so, it sounds more like a frontnend feature to me? Who will benefit with this feature? All tensorboard users.

Any Other info.

psybuzz commented 4 years ago

Thank you for the report. I've anecdotally heard this request from another user in-person, so it seems worth investigation. I think some research is required to better understand the use cases. e.g.

do users expect to view more than one subgroup at a time?
we see many users with 2 - 15 top-level tag groups, but do current logs have 1-2 subgroups, or do we expect to see 10+?
are there common cases where '/' in a tag name should not create a subgroup?
do users expect deeply nested subgroups within subgroups?
is it useful to view multiple subgroups at once, or is one at a time sufficient?
why does the existing tag filter field not fully address this?

As an optional step to help us better understand your particular setup, would you be interested in sharing your tag names? If they are not sensitive, I think the structure of tag names (how many subgroups, how deeply they might nest) might be a useful data point.

To collect all the tag names, you can run this in the browser devtools console: [...new Set(Object.values(await (await fetch("http://localhost:6006/data/plugin/scalars/tags")).json()).map(Object.keys).flat())]

alexcoca commented 4 years ago

Hi @psybuzz,

Thanks for getting back, apologies for being late in responding -- a busy spell at this end. Let me get back to you with a proper answer to your questions (hopefully tomorrow).

Regarding tag names - absolutely! The code is open source and currently work in progress but I will share tag names together with the logic behind them. You'll also be able to use my code to test any changes you might want to make. Chat soon!

Many thanks,

s-rog commented 3 years ago

Any updates or plans for this feature request?

wico-silva commented 2 years ago

I would also like to have this feature. I replied to your questions, @psybuzz, below:

do users expect to view more than one subgroup at a time?

I do, it should work exactly like it does now - I can expand or hide any group I want.

we see many users with 2 - 15 top-level tag groups, but do current logs have 1-2 subgroups, or do we expect to see 10+?

This varies a lot. I would say anything should be possible, it should be up to the user to decide how many they want for their particular application. For me this could be anything from 0 to something around 10, more than that and it would be hard to analyze anyway. Here's an example of tag names for an object detection use-case:

loss/train/all
loss/train/box
loss/train/class
loss/val/all
loss/val/box
loss/val/class
metrics/mean/<some metric 0>
metrics/mean/<some metric 1>
...
metrics/\/<some metric 0>
metrics/\/<some metric 1>
...

In this case, we would have:

2 subgroups inside the loss group
C + 1 subgroups inside the metrics group, where C is the number of classes

are there common cases where '/' in a tag name should not create a subgroup?

Can't think of one and tensorboard already groups using the first /. So it's already reserved for this purpose, sort of.

do users expect deeply nested subgroups within subgroups?

This could be configurable in the web view - the maximum depth allowed for subgrouping

is it useful to view multiple subgroups at once, or is one at a time sufficient?

I think this is the same as the first question.

why does the existing tag filter field not fully address this?

With multiple subgroups it's easier to compare multiple subgroups directly since they are stacked on the web page. The filter can provide something similar, but not quite - the plots are in a sequence and not stacked. Having to know or figure out the correct regex for the filter is also a worse UX.