ceccocats / tkDNN

Deep neural network library and toolkit to do high performace inference on NVIDIA jetson platforms
GNU General Public License v2.0
718 stars 208 forks source link

How to interface with python Torchrt models? #294

Open karanamrahul opened 2 years ago

perseusdg commented 1 year ago

Unfortunately this repo doesn't support TorchRt models .You would need to manually export the weights from the pytorch model to layer wise weights and then write the model again using tkdnn layers,if your model has a cfg file like those found in yolo models and if it has only layers that are found in tkdnn , you can use the darknet parser to parse through the cfg file and return a model to you ,but you would still need to export the model weights to layer wise weights.You can have a look at the repos mentioned in exporting_weights.md file to get a general idea on how to export your model weigths to a tkdnn acceptable format and you can have a look at the example in the tests folder to get an idea on how to write the model using tkdnn layers or use the darknet parser.Hope this helps you out :)