adityagoel4512
commented
3 months ago One broader note/thought on this. The type gymnastics required to ensure things work smoothly on the onnxruntime backend is really best suited for a separate package entirely (which we may or may not import here) that consolidates the rewrite rules being performed at the graph level.
A few reasons:
- It means you can consolidate the techniques required and unit test this more systematically. You might even be able to automate much of the kernel diffing.
- It is not exclusively a challenge for ndonnx.
- It simplifies this codebase dramatically.
- There is no clear timeline when ndonnx should remove any of these "hacks" once upstream changes are made but a rewrite library with a versioned API would make it possible to specify this explicitly.
- onnxruntime isn't the only backend. Many of these workarounds simply are not relevant for other backends. Ultimately this is a library that should just focus on standard compliant ONNX export.
The next release of onnxruntime will sport some more kernels that we need for reduction operations (related PR). Let's revisit some of our data type work around after the next release.