search

styler00dollar / VSGAN-tensorrt-docker

Using VapourSynth with super resolution and interpolation models and speeding them up with TensorRT.
BSD 3-Clause "New" or "Revised" License
286 stars 30 forks source link

Why "RIFE tensorrt not possible" ? #11

Closed Justin62628 closed 2 years ago

Justin62628 commented 2 years ago

At line 8 of inference.py, it is mentioned that "RIFE tensorrt is not possible" I wonder the reason as this note lacks explanation

styler00dollar commented 2 years ago

TensorRT only supports a subset of operations and has onnx has intermediate. At the time of adding that, grid_sampler was not a valid onnx op, so it wouldn't work to export as onnx easily. On top of that there wasn't official grid_sampler support by TensorRT. Very recently onnx added grid_sampler into onnx opset 16, which you can only use in PyTorch if you use a nightly or compile yourself currently. TensorRT mentioned in an issue that they plan on adding this op at some point. Currently there is no easy way to get grid_sampler working, but that hopefully changes at some point.

An alternative would be to add a custom TensorRT plugin, if you are willing to add that manually and recompile TensorRT, but I couldn't manage to do so. I am waiting for official support.

Justin62628 commented 2 years ago

That's a thorough explanation, thanks!