search

dusty-nv / ros_deep_learning

Deep learning inference nodes for ROS / ROS2 with support for NVIDIA Jetson and TensorRT
887 stars 258 forks source link

Increase net batch-size #75

Open r91andersson opened 3 years ago

r91andersson commented 3 years ago

Is it planned in near future to improve the jetson-inference to support net with batch-sizes larger than 1 (that is current maximum)? @dusty-nv How much work would it be to fix such a feature? Would you be able to point out where to start dig in the code? Or do you think that would be to complex?

The background for this is that we have a SSDMobileNetV2 that runs at approx. 65 fps (so basically it handles 2 camera streams at 30 fps). We would like to run 6 cameras, this means that we would need total 3 Xavier AGX to run that setup.

But if it would be possible to increase the batch size to 6 instead, we could have all cameras capturing images on a synced trigger event, then we could collect those and make a batch of 6 images and run them on one Xavier instead.

dusty-nv commented 3 years ago

I don't currently have plans to do it in jetson-inference - for multi-camera setup, I would recommend at looking at DeepStream which has optimized performance for multi-camera streaming+inference, and you can also train your model with Transfer Learning Toolkit (TLT) which can prune your model and further improve the performance.

r91andersson commented 3 years ago

@dusty-nv Ok, thanks for info!