Metrics from today do not show in Trends

[ivyleavedtoadflax]

I have a model run from today at 09:33 which which is not showing up in the trend plots when i select '1 week', '1 month' or '1 year':

It does appear when i select 'all time' (see brown dot to the far right):

[Suor]

I also see that long branch names are displayed poorely.

[ivyleavedtoadflax]

Additionally if you have the same metric name e.g. weighted avg.f1-score in multiple columns, it's not clear which plot refers to which column.

[Suor]

Real data always challenges the initial design. I guess we'll need to make labels smart to show differing parts, although we still have a bit vertical space in your case to simply prefix them.

[arcticbear]

Additionally if you have the same metric name e.g. weighted avg.f1-score in multiple columns, it's not clear which plot refers to which column.

@ivyleavedtoadflax Thank you for noticing that and for the feedback! What additional information in the plot labels would help to make that clearer for you? For example, file name as a prefix for the metric name, would it works?

[arcticbear]

@iterative/studio-frontend, we should fix alignments between checkboxes and their text labels in case of extremely long texts, see example below:

Example

[ivyleavedtoadflax]

Additionally if you have the same metric name e.g. weighted avg.f1-score in multiple columns, it's not clear which plot refers to which column.

@ivyleavedtoadflax Thank you for noticing that and for the feedback! What additional information in the plot labels would help to make that clearer for you? For example, file name as a prefix for the metric name, would it works?

No worries :+1: Enjoying working with it!

I guess the sensible thing would to be to refer to to the file from which the metrics came.

One a side note it would also be great to be able to resize the trends window to see the plots larger.

[arcticbear]

@ivyleavedtoadflax I'm just wondering because we're already working on those improvements and features. Every little detail 'from the fields' might be helpful and valuable. Thank you for the feedback and suggestions!

[ivyleavedtoadflax]

This looks like it is fixed now @arcticbear, thanks! I'll keep you posted on any additional issues I run into.

[jellebouwman]

@ivyleavedtoadflax I've been looking at the initial bug you reported: metrics from experiments run today are only showing up in the All time view. I've been able to reproduce this behavior, but in a very specific use case.

1. I run an experiment from the Studio application, and add the commit to a new branch.
2. While waiting on the experiment, I do not close the app
3. After the experiment closes, I open the Trends panel, and only select my new new branch.
4. When I change the time range from anything but All Time, I get the No data to show for this time range notification.
5. When I reload the page, and reopen the Trends panel, again only selecting my new branch, I do see plot information for those commits.

This is a bug, and I'm planning to fix it.

I'm curious if this might be the same scenario you originally ran into when creating this ticket. Could you, if you can remember, confirm this behavior?

[ivyleavedtoadflax]

Hi @jellebouwman, that sounds incredibly specific. These are my recollections of how I experienced the big:

1. I run an experiment from the Studio application, and add the commit to a new branch.

No, in my case I was launching the experiments using the CML github action with an AWS runner.

2. While waiting on the experiment, I do not close the app

Yes, this is likely.

3. After the experiment closes, I open the Trends panel, and only select my new new branch.

No, I was selecting just the main and the new branch as shown in the images I posted above.

4. When I change the time range from anything but All Time, I get the No data to show for this time range notification.

No, I did not receive the No data to show message

5. When I reload the page, and reopen the Trends panel, again only selecting my new branch, I do see plot information for those commits.

No, I failed to see the information for the new commits also when I had the main branch selected, as above.

I hope this is helpful. It does seem to have resolved, so perhaps there was something quite specific about the sequence of actions I had made, but my experience does not match well with what you describe.

[jellebouwman]

Thank you for the elaborate response @ivyleavedtoadflax! Seems like the bug I described pertains to a different edge case. I have opened an internal ticket for this where we will address this.

I hope this is helpful. It does seem to have resolved, so perhaps there was something quite specific about the sequence of actions I had made, but my experience does not match well with what you describe.

In regards to this ticket, if your specific issue has been resolved, I think we should close this ticket for now. Please, feel free to re-open this if you run into this again or have some different feedback!

[shcheklein]

@ivyleavedtoadflax hey, to clarify - did you run an experiment and waited with an app and the trends panel open?

@jellebouwman can it be related to us now updating it in "real-time"?

[ivyleavedtoadflax]

@ivyleavedtoadflax hey, to clarify - did you run an experiment and waited with an app and the trends panel open?

I think it was something I looked at after the experiments had already run, but I guess its possible I had a studio tab open with the trends panel open while the experiment was running. It takes over an hour to run, so I certainly wasn't sitting and waiting for it.

[jellebouwman]

@ivyleavedtoadflax you mentioned the error has been fixed for you. A fix for the scenario I described has just been released. I'm closing this ticket as I think all the actions have been completed, but, please, re-open or create a new ticket if you run into anything! Thanks for your feedback and suggestions!