search

mlcommons / cm4mlops

A collection of portable, reusable and cross-platform automation recipes (CM scripts) to make it easier to build and benchmark AI systems across diverse models, data sets, software and hardware
http://docs.mlcommons.org/cm4mlops/
Apache License 2.0
14 stars 20 forks source link

Submission generation Failure - Generate system_meta.json through CM #522

Closed anandhu-eng closed 1 week ago

anandhu-eng commented 2 weeks ago

Submission generation is failing if system_meta.json is not present anywhere in results directory. Would it be great if CM could generate one while submission generation.

Error:

INFO:root:       ! call "postprocess" from /home/anandhu/CM/repos/mlcommons@cm4mlops/script/preprocess-mlperf-inference-submission/customize.py
INFO:root:* cm run script "submission inference checker mlc"
INFO:root:  * cm run script "get python3"
INFO:root:       ! load /home/anandhu/CM/repos/local/cache/8623b1ecafb64fea/cm-cached-state.json
INFO:root:Path to Python: /usr/bin/python3
INFO:root:Python version: 3.10.12
INFO:root:  * cm run script "get mlcommons inference src"
INFO:root:       ! load /home/anandhu/CM/repos/local/cache/5775adcf16bc4d93/cm-cached-state.json
INFO:root:  * cm run script "get generic-python-lib _xlsxwriter"
INFO:root:       ! load /home/anandhu/CM/repos/local/cache/8cd70dc657cb45cc/cm-cached-state.json
INFO:root:  * cm run script "get generic-python-lib _package.pyarrow"
INFO:root:       ! load /home/anandhu/CM/repos/local/cache/42c52a66324f4c17/cm-cached-state.json
INFO:root:  * cm run script "get generic-python-lib _pandas"
INFO:root:       ! load /home/anandhu/CM/repos/local/cache/2c0956ca5c7043f0/cm-cached-state.json
/usr/bin/python3 '/home/anandhu/CM/repos/local/cache/cb3c42e4cfe24f62/inference/tools/submission/submission_checker.py' --input '/home/anandhu/CM/repos/local/cache/9f8d333784a5499f/mlperf-inference-submission' --submitter 'MLCommons' 
INFO:root:       ! cd /home/anandhu
INFO:root:       ! call /home/anandhu/CM/repos/mlcommons@cm4mlops/script/run-mlperf-inference-submission-checker/run.sh from tmp-run.sh
/usr/bin/python3 '/home/anandhu/CM/repos/local/cache/cb3c42e4cfe24f62/inference/tools/submission/submission_checker.py' --input '/home/anandhu/CM/repos/local/cache/9f8d333784a5499f/mlperf-inference-submission' --submitter 'MLCommons' 
Traceback (most recent call last):
  File "/home/anandhu/CM/repos/local/cache/cb3c42e4cfe24f62/inference/tools/submission/submission_checker.py", line 3162, in <module>
    sys.exit(main())
  File "/home/anandhu/CM/repos/local/cache/cb3c42e4cfe24f62/inference/tools/submission/submission_checker.py", line 2997, in main
    results, systems = check_results_dir(
  File "/home/anandhu/CM/repos/local/cache/cb3c42e4cfe24f62/inference/tools/submission/submission_checker.py", line 2074, in check_results_dir
    available = system_json.get("status").lower()
AttributeError: 'NoneType' object has no attribute 'lower'