Add DLRM_DCNV2 and GPT3 names to logging repo

mlcommons / logging

MLPerf™ logging library

https://mlcommons.org/en/groups/best-practices-benchmark-infra

Apache License 2.0

30 stars 45 forks source link

Add DLRM_DCNV2 and GPT3 names to logging repo #306

Closed ShriyaPalsamudram closed 1 year ago

ShriyaPalsamudram commented 1 year ago

_ALL_RESULT_FILE_COUNTS and _ALL_ALLOWED_BENCHMARKS updated to reflect new benchmarks (DLRM_DCNv2 and GPT3) File counts for DLRM_DCNv2 set to 10, GPT3 set to 3

@pgmpablo157321 should we update information for the new benchmarks anywhere else so the package checker and result summarizer are happy during Training 3.0?

cc @sgpyc @janekl

github-actions[bot] commented 1 year ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaPalsamudram commented 1 year ago

@itayhubara - can you please approve this?

pgmpablo157321 commented 1 year ago

@maanug-nv I think you need to update these files as well: https://github.com/mlcommons/logging/blob/0376a400b8c52dd6be6aef9b63d6ae29bbf72349/mlperf_logging/compliance_checker/training_3.0.0/open_common.yaml#L5 https://github.com/mlcommons/logging/blob/0376a400b8c52dd6be6aef9b63d6ae29bbf72349/mlperf_logging/compliance_checker/training_3.0.0/closed_common.yaml#L5 and I think here as well: https://github.com/mlcommons/logging/blob/0376a400b8c52dd6be6aef9b63d6ae29bbf72349/mlperf_logging/mllog/constants.py#L40-L50

ShriyaPalsamudram commented 1 year ago

@pgmpablo157321 made those changes, please review.

janekl commented 1 year ago

I think we need a couple of extra changes from https://github.com/mlcommons/logging/pull/309. But to close all this in a single PR we could add the changes here.

Specifically I mean:

renaming https://github.com/mlcommons/logging/blob/master/mlperf_logging/rcp_checker/training_3.0.0/rcps_dlrmv2.json -> rcps_dlrm_dcnv2.json
renaming all keys here https://github.com/mlcommons/logging/blob/master/mlperf_logging/rcp_checker/training_3.0.0/rcps_dlrmv2.json#L3 to dlrm_dcnv2_ref_32768 (there are three such keys)
setting the correct benchmark name dlrm_dcnv2 here (also three times) https://github.com/mlcommons/logging/blob/master/mlperf_logging/rcp_checker/training_3.0.0/rcps_dlrmv2.json#L4
renaming https://github.com/mlcommons/logging/blob/master/mlperf_logging/compliance_checker/training_3.0.0/closed_dlrmv2.yaml to closed_dlrm_dcnv2.yaml
renaming https://github.com/mlcommons/logging/blob/master/mlperf_logging/compliance_checker/training_3.0.0/open_dlrm.yaml to open_dlrm_dcnv2.yaml

janekl commented 1 year ago

As for DLRM_DCNv2, all the changes needed are here now, so I'll close the other PR https://github.com/mlcommons/logging/pull/309, thanks Shriya.

I think this is ready to be merged now, please merge @pgmpablo157321

ShriyaPalsamudram commented 1 year ago

@pgmpablo157321 - can you please approve and merge this PR?