search

microsoft / superbenchmark

A validation and profiling tool for AI infrastructure
https://aka.ms/superbench
MIT License
269 stars 59 forks source link

Analyzer - Generate baseline given results from multiple nodes. #573

Closed guoshzhao closed 11 months ago

guoshzhao commented 11 months ago

Description Input is jsonline file with results from multiple nodes, generate baseline file according to customized rules.

codecov[bot] commented 11 months ago

Codecov Report

Attention: 220 lines in your changes are missing coverage. Please review.

Comparison is base (97c9a41) 87.33% compared to head (6ded588) 79.65%. Report is 32 commits behind head on main.

Files Patch % Lines
superbench/analyzer/generate_baseline.py 0.00% 127 Missing :warning:
superbench/analyzer/generate_statistic.py 0.00% 92 Missing :warning:
superbench/analyzer/file_handler.py 0.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #573 +/- ## ========================================== - Coverage 87.33% 79.65% -7.68% ========================================== Files 89 95 +6 Lines 5946 7045 +1099 ========================================== + Hits 5193 5612 +419 - Misses 753 1433 +680 ``` | [Flag](https://app.codecov.io/gh/microsoft/superbenchmark/pull/573/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | Coverage Δ | | |---|---|---| | [directx-unit-test](https://app.codecov.io/gh/microsoft/superbenchmark/pull/573/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `34.40% <0.00%> (?)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

guoshzhao commented 11 months ago

@cp5555 This branch is used by many teams, to make sure their migration be safe, will not submit commits anymore to this branch. Please review the new one: https://github.com/microsoft/superbenchmark/pull/575