Open GowthamKudupudi opened 3 years ago
The current tkdnn doesn't support plugins registry (which is why it doesn't work on trt8). You can try building the .so in this branch: https://github.com/TheExDeus/tkDNN/tree/feature/tensorrt8_support
TensorRT8 is now supported on tensorrt8 branch.
I have generated a
.rt
(.plan
) file of a trainedyolo4tiny
Darknet model by passing theweights
,input_bins
andout_bins
totests/darknet/yolo4tiny.cpp
.When I try to serve the generated
.plan
file with Triton server, it fails to load the model with error:When I try to run
Triton
withLD_PRELOAD=libkernels.so
it throws the below additional errorshow to create custom plugins
.so
with tkDNN? islibkernels.so
that was created while building thetkDNN
, the custom plugins shared object? Can I generate the.rt
(.plan
) file by using inbuiltTRT
plugins mentioned in the above log in place of custom plugins?I have built and run the
tritonserver
on the same container in whichtkDNN
is built and.rt
file is generated.