search

mlcommons / inference

Reference implementations of MLPerf™ inference benchmarks
https://mlcommons.org/en/groups/inference
Apache License 2.0
1.19k stars 519 forks source link

Optional field for closed division marked as required #1444

Open nvyihengz opened 1 year ago

nvyihengz commented 1 year ago 
[2023-07-16 01:20:48,197 submission_checker.py:2872 ERROR] closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx1_TRT, field host_memory_configuration requires a meaningful response but is empty
[2023-07-16 01:20:48,197 submission_checker.py:2872 ERROR] closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx1_TRT, field host_networking requires a meaningful response but is empty
[2023-07-16 01:20:48,197 submission_checker.py:2872 ERROR] closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx1_TRT, field host_networking_topology requires a meaningful response but is empty
[2023-07-16 01:20:48,197 submission_checker.py:2872 ERROR] closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx1_TRT, field accelerator_host_interconnect requires a meaningful response but is empty
[2023-07-16 01:20:48,197 submission_checker.py:2872 ERROR] closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx1_TRT, field accelerator_interconnect requires a meaningful response but is empty
[2023-07-16 01:20:48,197 submission_checker.py:2872 ERROR] closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx1_TRT, field cooling requires a meaningful response but is empty
[2023-07-16 01:20:48,198 submission_checker.py:2865 ERROR] closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx1_TRT, field host_networking_card_count is missing
[2023-07-16 01:20:48,198 submission_checker.py:2865 ERROR] closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx1_TRT, field system_type_detail is missing

These field are reported as missing by the submission checker.

arjunsuresh commented 1 year ago

@nvyihengz All these fields are mandatory as per the rules here. Except system_type_detail all others need a meaningful response. But if any field does not make sense on a given system, "N/A" is enough to satisfy the submission checker.