Integrate with ONNX 1.16.0 release branch

[cjvolzka]

Describe the feature request

We are releasing ONNX 1.16.0. A release branch is created (https://github.com/onnx/onnx/tree/rel-1.16.0). The planned release date is March 25, 2024. Release candidates are also available from TestPyPI: pip install -i https://test.pypi.org/simple/ --pre onnx

It is important to integrate ONNX release branch into ORT ASAP so that any issues and incompatibilities can be detected and resolved before the ONNX release.

Please follow instructions at (https://github.com/microsoft/onnxruntime/blob/master/docs/How_To_Update_ONNX_Dev_Notes.md) to integrate with the ONNX release branch. Please implement CPU kernels for new and updated ONNX ops. A list of new and updated ops can be found at (https://github.com/onnx/onnx/wiki/Logistics-for-ONNX-Release-1.16.0).

Key updates:

• ai.onnx Opset 21
  • Update to support int4 and uint4:
    • Cast, CastLike, Constant, ConstantOfShape, Identity, If, Loop, Reshape, Scan, Shape, Size
    • Update to support float8e4m3fnuz, float8e5m2, float8e5m2fnuz, int4 and uint4:
    • Flatten, Pad, Squeeze, Transpose, Unsqueeze
  • Support blocked quantization. Support int4, uint4, int16, and uint16:
    • DequantizeLinear, QuantizeLinear,
  • Support bfloat16 and float16 scales. Support float8e4m3fn, float8e4m3fnuz, float8e5m2, float8e5m2fnuz quantized tensors:
    • QLinearMatMul,
  • Add stash_type attribute to GroupNormalization
• ai.onnx.ml Opset 5
  • Addeded new operator TreeEnsemble
  • Deprecated TreeEnsembleClassifier and TreeEnsembleRegressor
• IR Version 10
  • Enabled UINT4, INT4 support
• Support Python 3.12

In case a bug in ONNX is detected during integration of ONNX 1.16.0, please open a ONNX Bug Report and tag ONNX Release Manager @cjvolzka so that the bug is fixed in the ONNX release branch.

Describe scenario use case

ORT is integrated with ONNX rel-1.16.0 with all CI pipelines passing.

[cjvolzka]

Related PR: https://github.com/microsoft/onnxruntime/pull/19745

[jkosek]

Is there any timeline for support of onnx 1.16.0 and IR 10 in onnxruntime?

[EwoutH]

Since #19745 is merged, is this issue fully completed?

[cjvolzka]

Since https://github.com/microsoft/onnxruntime/pull/19745 is merged, is this issue fully completed?

It appears so. @liqunfu, was everything covered or is there something noteworthy to keep this issue open?