Closed lizexu123 closed 10 months ago
According to this it should be possible, but you'd need to create a IExecutionContext
for each stream. In the case of my code, you'd basically just create an instance of the Engine
class for each model you want to run in parallel. However, do realize that you will have GPU contention if running models in parallel and not in serial.
Hello, does enqueueV3 support multi-stream inference? Is this part included in your code?