AutoResearch / autodoc

MIT License
0 stars 1 forks source link

perf: improve logging for MLflow metrics #42

Closed anujsinha3 closed 7 months ago

anujsinha3 commented 7 months ago

Change Description

41 - Improving the MLflow to handle (prevent overriding) metrics logging when running eval() or eval_prompts() with multiple prompts.

Example Run: https://ml.azure.com/experiments/id/e1882f37-87e8-4ff9-930a-7aa050693072/runs/gray_coat_sy48gpw0jc?wsid=/subscriptions/e4f4cccd-ac89-41b8-aae6-896e5a8471e4/resourcegroups/AutoRADoc/providers/Microsoft.MachineLearningServices/workspaces/AutoRAML&tid=f6b6dd5b-f02f-441a-99a0-162ac5060bd2#overview

Solution Description

Code Quality

Project-Specific Pull Request Checklists

Bug Fix Checklist

New Feature Checklist

Documentation Change Checklist

Build/CI Change Checklist

Other Change Checklist

codecov-commenter commented 7 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (aac9880) 98.65% compared to head (8db2bc0) 98.63%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #42 +/- ## ========================================== - Coverage 98.65% 98.63% -0.03% ========================================== Files 8 8 Lines 298 292 -6 ========================================== - Hits 294 288 -6 Misses 4 4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

anujsinha3 commented 7 months ago

Also updated the code to use mlflow.logMetrics() - allows for multiple metrics logging via dictionary instead of log.metric() - which logs only one key-value pair at a time.