Look into TVM+sparsity

[vvolhejn]

https://tvm.apache.org/docs/how_to/tune_with_autoscheduler/tune_network_arm.html

https://tvm.apache.org/docs/how_to/deploy_models/deploy_sparse.html

[vvolhejn]

https://discuss.tvm.apache.org/t/running-a-cnn-using-sparsity-convertor/7267/7 says there is no performance improvement for CNNs, https://gist.github.com/Wheest/f608b7ce16b0b7c7a3414e950e934033 is the associated self-contained example

https://discuss.tvm.apache.org/t/sparse-conv2d-runs-slower-than-normal-conv2d/11430

Implementation of the optimizations: https://github.com/apache/tvm/tree/main/python/tvm/relay/data_dep_optimization

This is probably the right function to call, as used by the tune_network_arm tutorial: https://github.com/apache/tvm/blob/720e7b1ebd9b789a1100dee7536d0633c7941dd1/python/tvm/topi/sparse/utils.py#L143

relevant PR? https://github.com/apache/tvm/pull/8065

Rewriter https://github.com/apache/tvm/blob/248ad45c6ab197fbec11512b596c7449db2febe6/src/relay/transforms/convert_sparse_conv2d.cc#L184

[vvolhejn]

Got it to work for dense and inverted_bottleneck models, but for dilated_conv it does nothing (probably only works on 1x1 convs) and is broken for dilated_conv_ib: https://discuss.tvm.apache.org/t/invalid-blocksize-when-trying-to-convert-cnn-to-sparse/13016