ssec-jhu / dplutils

Distributed(Data, Discovery) Pipeline Uitilities
BSD 3-Clause "New" or "Revised" License
1 stars 0 forks source link

adding aim and mlflow observers #26

Closed amitschang closed 11 months ago

amitschang commented 11 months ago

This allows to track custom user metrics in one of a couple of popular tracking utilities commonly used in machine learning. Each tracks experiments and their runs and provides a UI and database of historical data. Here are examples of parts of UI from unit test runs:

codecov[bot] commented 11 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (c7a2ba4) 100.00% compared to head (3d74c12) 100.00%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #26 +/- ## ========================================= Coverage 100.00% 100.00% ========================================= Files 7 9 +2 Lines 218 265 +47 ========================================= + Hits 218 265 +47 ``` | [Flag](https://app.codecov.io/gh/ssec-jhu/dplutils/pull/26/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ssec-jhu) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/ssec-jhu/dplutils/pull/26/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ssec-jhu) | `100.00% <100.00%> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ssec-jhu#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

amitschang commented 11 months ago

I've updated based on you comment, thanks! While doing so I realized missed opportunity to test that code path, and then as a result of the test realized that mlflow check needed to be moved :smile: