search

iterative / dvclive

📈 Log and track ML metrics, parameters, models with Git and/or DVC
https://dvc.org/doc/dvclive
Apache License 2.0
160 stars 32 forks source link

integrations: ultralytics/yolo #603

Open dberenbaum opened 12 months ago

dberenbaum commented 12 months ago

Need to document the yolo callback from https://github.com/ultralytics/ultralytics/blob/main/ultralytics/yolo/utils/callbacks/dvc.py into dvclive and utralytics docs.

shcheklein commented 12 months ago

Ultralytics generates their docs for all the callbacks I think? Or do you want to add a more comprehensive overview, etc? I was thinking about this, but I don't feel we are ready for that. I don't know how to use the callback in the real project yet. (specifically, what do with images folder, how to combine it with the existing dvc.yaml, etc)- I felt we need some time to address those (or agree that those are not important / we have a good workaround).

daavoo commented 12 months ago

Ultralytics generates their docs for all the callbacks I think?

This is the only thing rendered, which is not very helpful I guess https://docs.ultralytics.com/reference/yolo/utils/callbacks/dvc/ Can we add some docstrings?

Or do you want to add a more comprehensive overview, etc?

I think we could start by adding a page to our docs https://dvc.org/doc/dvclive/ml-frameworks with minimum explanation on how to use it.

There is also this in the ultralytics docs: https://docs.ultralytics.com/modes/train/#logging but TBH I don't really understand the page (Maybe it is me but I haven't figured out yet how to set up / use any logger just form the docs)

I was thinking about this, but I don't feel we are ready for that. I don't know how to use the callback in the real project yet. (specifically, what do with images folder, how to combine it with the existing dvc.yaml, etc)- I felt we need some time to address those (or agree that those are not important / we have a good workaround).

I was also planning to try to use it in a project myself and see what is going on. Also, I see that images can be valuable but can we start by not using them? Other loggers don't do anything with them and I think it is ok to start with metrics and linear plots while we figure this other things

dberenbaum commented 12 months ago

For comparison: https://docs.wandb.ai/guides/integrations/yolov5

shcheklein commented 12 months ago

Also, I see that images can be valuable but can we start by not using them? Other loggers don't do anything with them and I think it is ok to start with metrics and linear plots while we figure this other things

All loggers actually do log them, and support per step navigation.

I think we could start by adding a page to our docs https://dvc.org/doc/dvclive/ml-frameworks with minimum explanation on how to use it.

Yes, I can start on that.

shcheklein commented 10 months ago

I think it's not done, right? We still don't have docs / guide on the YOLO side, let's keep it open for now?

shcheklein commented 10 months ago

Relevant https://github.com/ultralytics/ultralytics/pull/4343#issuecomment-1676447390

daavoo commented 9 months ago

Lowering to p2, as we have a decent enough integration and docs in what we can control (dvc.org), not sure what is left beyond docs/guide on YOLO side