How do ops under ML namespace enter the score calculation?

onnx / backend-scoreboard

Scoreboard for ONNX Backend Compatibility

http://onnx.ai/backend-scoreboard/

Apache License 2.0

24 stars 27 forks source link

How do ops under ML namespace enter the score calculation? #55

Open yuanyao-nv opened 1 year ago

yuanyao-nv commented 1 year ago

I see there have been new tests added for ops under the ml namespace, see this PR https://github.com/onnx/onnx/pull/4881.

The current scoreboard setup will automatically pick up all tests under the backend/test/case/node directory. Since most backend providers don't support the ml ops, should such tests be excluded from the scoreboard? eg. by specifying -k "not test_ai_onnx_ml_".

cbourjau commented 11 months ago

As a user of the ML opset I'd love to see them included :)