[TMVA][SOFIE] Status of the support of ONNX operators

[axmat]

This issue should track the status of the support of ONNX operators in TMVA/SOFIE.

Supported :

• [x] Add
• [x] AveragePool
• [x] BatchNormalization
• [x] Conv
• [x] Gemm
• [x] GlobalAveragePool
• [x] GRU
• [x] LSTM
• [x] MaxPool
• [x] Pool
• [x] Relu
• [x] Reshape
• [x] RNN
• [x] Selu
• [x] Sigmoid
• [x] Slice
• [x] Transpose

Work in progress :

• [ ] Softmax, #10315
• [ ] Identity, #10315
• [ ] Concat, #10315 and #9311
• [ ] InstanceNormalization, #8885

Missing :

• [ ] Abs
• [ ] Acos
• [ ] Acosh
• [ ] And
• [ ] ArgMax
• [ ] ArgMin
• [ ] Asin
• [ ] Asinh
• [ ] Atan
• [ ] Atanh
• [ ] BitShift
• [ ] Cast
• [ ] Ceil
• [ ] Clip
• [ ] Compress
• [ ] ConcatFromSequence
• [ ] Constant
• [ ] ConstantOfShape
• [ ] ConvInteger
• [ ] ConvTranspose
• [ ] Cos
• [ ] Cosh
• [ ] CumSum
• [ ] DepthToSpace
• [ ] DequantizeLinear
• [ ] Det
• [ ] Div
• [ ] Dropout
• [ ] Einsum
• [ ] Elu
• [ ] Equal
• [ ] Erf
• [x] Exp
• [ ] Expand
• [ ] EyeLike
• [ ] Flatten
• [ ] Floor
• [ ] Gather
• [ ] GatherElements
• [ ] GatherND
• [ ] GlobalLpPool
• [ ] GlobalMaxPool
• [ ] Greater
• [ ] GridSample
• [ ] HardSigmoid
• [ ] Hardmax
• [ ] Identity
• [ ] If
• [ ] IsInf
• [ ] IsNaN
• [ ] LRN
• [ ] LeakyRelu
• [ ] Less
• [ ] Log
• [ ] Loop
• [ ] LpNormalization
• [ ] LpPool
• [ ] MatMul
• [ ] MatMulInteger
• [ ] Max
• [ ] MaxRoiPool
• [ ] MaxUnpool
• [ ] Mean
• [ ] Min
• [ ] Mod
• [ ] Mul
• [ ] Multinomial
• [ ] Neg
• [ ] NonMaxSuppression
• [ ] NonZero
• [ ] OneHot
• [ ] Optional
• [ ] OptionalGetElement
• [ ] OptionalHasElement
• [ ] Or
• [ ] PRelu
• [ ] Pad
• [ ] Pow
• [ ] QLinearConv
• [ ] QLinearMatMul
• [ ] QuantizeLinear
• [ ] RandomNormal
• [ ] RandomNormalLike
• [ ] RandomUniform
• [ ] RandomUniformLike
• [ ] Reciprocal
• [ ] ReduceL1
• [ ] ReduceL2
• [ ] ReduceLogSum
• [ ] ReduceLogSumExp
• [ ] ReduceMax
• [ ] ReduceMean
• [ ] ReduceMin
• [ ] ReduceSum
• [ ] ReduceSumSquare
• [ ] Resize
• [ ] ReverseSequence
• [ ] RoiAlign
• [ ] Round
• [ ] Scan
• [ ] ScatterElements
• [ ] ScatterND
• [ ] SequenceAt
• [ ] SequenceConstruct
• [ ] SequenceEmpty
• [ ] SequenceErase
• [ ] SequenceInsert
• [ ] SequenceLength
• [ ] Shape
• [ ] Shrink
• [ ] Sign
• [ ] Sin
• [ ] Sinh
• [ ] Size
• [ ] Softplus
• [ ] Softsign
• [ ] SpaceToDepth
• [ ] Split
• [ ] SplitToSequence
• [ ] Sqrt
• [ ] StringNormalizer
• [ ] Sub
• [ ] Sum
• [ ] Tan
• [ ] Tanh
• [ ] TfldfVectorizer
• [ ] ThresholdedRelu
• [ ] Tile
• [ ] TopK
• [ ] Trilu
• [ ] Unique
• [ ] Unsqueeze
• [ ] Where
• [ ] Xor

[youlxs]

Do we have the plan to support "MatMul" ?

[lmoneta]

MatMul is now supported in both master and 6.30, see

https://github.com/root-project/root/pull/12894

This above list needs to be updated with the new supported operator

[youlxs]

Do we have a plan to support "Split" ?

[lmoneta]

We can plan to support it if needed. It might take a little bit of time, since we need to support in SOFIE sequence of tensors which are the output of Split. For which type of model do you need Split?

[youlxs]

MatMul is now supported in both master and 6.30, see

12894

This above list needs to be updated with the new supported operator

thanks

[youlxs]

We can plan to support it if needed. It might take a little bit of time, since we need to support in SOFIE sequence of tensors which are the output of Split. For which type of model do you need Split?

I want to convert my model to the raw code to improve the prediction speed. As I saw from your slide the raw code generated through tmva has better performance than the original onnx runtime.