jatinchowdhury18
commented
1 year ago Hi there! At the moment, I haven't done any speed comparisons between RTNeural and inferencing engines that run on GPUs (or other hardware devices). With real-time audio systems, there are some constraints that can make it difficult to process real-time signals on a GPU, although there has recently been some new and exciting progress made in that area (see, e.g. this paper, or the things that GPU Audio is working on). Personally, I haven't tried experimenting with real-time processing on the GPU just yet.
That said, it probably would be useful to do some performance comparison measurements with inferencing engines that run on a GPU, particularly for the size of networks that I usually use with RTNeural. I imagine that information might be interesting for folks outside of the audio realm as well!
TinaChen95
Hi, just wondering if you know from the top of your head if RTNeural runs faster than the accelerated M1 Pytorch mode or faster than using TensorRT on NVIDIA GPUs? I wanna test out if I can make a super fast Transformer to use with the GuitarML repos, which for now are using RTNeural as the speedy inference engine. Would be cool to try whatever complicated architecture super fast on Macbooks so neural plugins work with Logic.
Accelerated M1 PyTorch: https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/ NVIDIA TensorRT: https://docs.nvidia.com/deeplearning/tensorrt/index.html