search

iterative / studio-support

❓ DVC Studio Issues, Question, and Discussions
https://studio.iterative.ai
16 stars 1 forks source link

DVC tracked images don't appear in Studio #91

Closed CptCaptain closed 1 year ago

CptCaptain commented 1 year ago

I use DVC, Studio and DVCLive in my project (based on the ultralytics repo). With the cache_images feature of DVCLive enabled, the images that are produced are tracked in DVC. This causes Studio to throw lots of errors like this An error has occurred: dvclive/dvc.yaml::dvclive/plots/images/val/P_curve.png. Data is absent or failed to parse, and consequently fails to display them. Am I missing something, or is this just not supported (yet)? This looks very wrong, every commit is flagged as containing big remote files that were skipped and I need to scroll past a wall of errors to get to the plots each time, but I'd prefer tracking these frequently changing images via DVC over git.

shcheklein commented 1 year ago

@CptCaptain most likely it means that you need to setup a remote storage for DVC. Studio itself doesn't provide it out of the box. Do you use VS Code extension to setup the project?

Here is the https://dvc.org/doc/user-guide/data-management/remote-storage .

Please let me know if that makes sense, or if you need help with some setup here (or eve deciding on which storage to pick).

CptCaptain commented 1 year ago

Thanks for the quick response @shcheklein. I've already setup a google Bucket as remote storage, and I (or rather, CML) push the images there after training. It's also already setup in Studio and seems to work. The model checkpoints are also tracked by DVC, and they appear in Studio just fine.

shcheklein commented 1 year ago

@CptCaptain hey, it's great to hear that. Can we consider this one solved then? Or do you still see some issues?

CptCaptain commented 1 year ago

I'm sorry, maybe I was unclear. I was using the remote storage right from the start. And while the model checkpoints do appear just fine, the images do not. I just mentioned the checkpoints as that leads me to believe that the issue is not related to Studio not being able to access the storage location.

shcheklein commented 1 year ago

@CptCaptain hey, nw, thanks for clarifying this. Would you be able to share a screenshot of this errors (can be a direct message if this thread is not good - ivan at iterative.ai, also the repo URL). I can check the logs and see what is going on. Also, what is the size of the directory with plots in one of those commits, and size of those images with plots?

CptCaptain commented 1 year ago

@shcheklein I sent you a mail with the screenshot and URL. The directory is 13MB, and the individual image files are all <=700kB. The images with plots are even smaller, between 84kB and 244kB.

shcheklein commented 1 year ago

hey @CptCaptain , thanks, I'm looking into this.

shcheklein commented 1 year ago

@CptCaptain could you please reparse the repository (there is an option in the project dropdown menu Force Import). And see if it works now.

CptCaptain commented 1 year ago

@shcheklein, it works fine now :+1: Thanks for fixing it!