mlcommons / tiny_results_v1.0

This repository contains the results and code for the MLPerf™ Tiny Inference v1.0 benchmark.
https://mlcommons.org/en/inference-tiny-10/
Apache License 2.0
5 stars 1 forks source link

How to solve TypeError: unhasable type: 'dict' ? #1

Closed sahelib25 closed 1 year ago

sahelib25 commented 1 year ago

I am trying to reproduce the results for tiny v1.0 submitted by OctoML.

saheli@chai:~$ cm run script --tags=generate,tiny,mlperf,octoml,submission

* Searching for collective script(s) with tags "generate,tiny,mlperf,octoml,submission"
  - Number of scripts found: 1
  - Found script::wrapper-reproduce-octoml-tinyml-submission,b946001e289c4480 in /home/saheli/CM/repos/mlcommons@ck/cm-mlops/script/wrapper-reproduce-octoml-tinyml-submission
    - Requested version:  == {'r1.0': {}}
Traceback (most recent call last):
  File "/home/saheli/.local/bin/cm", line 8, in <module>
    sys.exit(run())
  File "/home/saheli/.local/lib/python3.10/site-packages/cmind/cli.py", line 35, in run
    r = cm.access(argv, out='con')
  File "/home/saheli/.local/lib/python3.10/site-packages/cmind/core.py", line 493, in access
    r = action_addr(i)
  File "/home/saheli/CM/repos/mlcommons@ck/cm-mlops/automation/script/module.py", line 832, in run
    if version!='' and version in versions:
TypeError: unhashable type: 'dict'
arjunsuresh commented 1 year ago

Hi @sahelib25 Thank you for filing the issue. This is due to an unfortunate typo which is fixed now. Can you please do

cm pull repo mlcommons@ck

and retry the commands?

gfursin commented 1 year ago

Thank you very much for your feedback, @sahelib25 !

By the way, please consider joining the MLCommons taskforce on education and reproducibility - we always welcome such feedback to continuously improve CM framework and help the community automate design space exploration of ML Systems!

We have added CM tests for MLPerf inference but did not have time to add tests for TinyMLPerf - we will discuss it during our next meeting in January!

Happy holidays!

gfursin commented 1 year ago

I have merged the fix from @arjunsuresh with the mainline mlcommons@ck .

@sahelib25 , please do not forget to update the main automation repository and try again:

cm pull repo mlcommons@ck

Thanks!

sahelib25 commented 1 year ago

Thank you @arjunsuresh and @gfursin for the fix.