appreciate your great work!
In our case, we use Deepstrean-Yolo to convert the model to deploy Yolov7 in Tensorrt. But I want to know if I have a live video with 25FPS (like traffic live video), can I use multiple batch size models like 4 batch sizes in this live video? In fact, if the real-time video is 25fps, it means that it takes 40ms for us to get one frame. If I use a model with a batch size of 4, I need to wait 160 milliseconds for each inference (I don't know if I understand this correctly). If this is the case, can I not use multi-batch-size models? We are currently building a pipeline using a model for each video channel. Can we take advantage of the fast multi-batch-size inference?
I hope you can help me answer this confusion, thank you very much!
appreciate your great work! In our case, we use Deepstrean-Yolo to convert the model to deploy Yolov7 in Tensorrt. But I want to know if I have a live video with 25FPS (like traffic live video), can I use multiple batch size models like 4 batch sizes in this live video? In fact, if the real-time video is 25fps, it means that it takes 40ms for us to get one frame. If I use a model with a batch size of 4, I need to wait 160 milliseconds for each inference (I don't know if I understand this correctly). If this is the case, can I not use multi-batch-size models? We are currently building a pipeline using a model for each video channel. Can we take advantage of the fast multi-batch-size inference? I hope you can help me answer this confusion, thank you very much!