Set specific Cuda stream ID in Cpp implimentation code.

wang-xinyu / tensorrtx

Implementation of popular deep learning networks with TensorRT network definition API

MIT License

7.03k stars 1.78k forks source link

Set specific Cuda stream ID in Cpp implimentation code. #1565

Open phamthanhdat270198 opened 3 months ago

phamthanhdat270198 commented 3 months ago

Env

Agx Orin 64GB.
Ubuntu20.04
Cuda version 11.4
TensorRT version 8.5.2

Question

Hi, I want to set specific cudastream ID on Cpp code yolo like --streams=0 or --streams=1, ... in trtexec Many thanks!!!!!

wang-xinyu commented 3 months ago

You need to adapt the inference code, create multiple streams and pass to infer() function.

phamthanhdat270198 commented 3 months ago

When I create multiple streams It just decrease a little bit infer time. So Does it working in parallel. give me an example about multi streams and multi contexts. Many thanks!!! Here is my impliment multi streams ![Uploading image.png…]()

stale[bot] commented 2 weeks ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.