Optimize PyTorch for GPU

[cczarnuch]

Be able to auto detect gpu and use that for the detections/tracking.

[cczarnuch]

Change object tracker to auto detect GPU

[icecap360]

Images on GPU: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-image.md TensorRT on Jetson Nano, includes trt conversion simple: https://gilberttanner.com/blog/run-pytorch-models-on-the-jetson-nano-with-tensorrt, TensorRT on Jetson Nano, complicated: https://rahulrav.com/blog/tensorrt_nano.html Good discussion on torch and GPU's: https://discuss.pytorch.org/t/solved-make-sure-that-pytorch-using-gpu-to-compute/4870/4

Others

Sending model to cuda/gpu: https://medium.com/@heldenkombinat/image-recognition-with-pytorch-on-the-jetson-nano-fd858a5686aa Torch to TRT converter: https://github.com/NVIDIA-AI-IOT/torch2trt General pytorch GPU examples: end to end https://pythonprogramming.net/gpu-deep-learning-neural-network-pytorch/ , https://medium.com/ai%C2%B3-theory-practice-business/use-gpu-in-your-pytorch-code-676a67faed09 Loading one of jetsons models instead of yolov5: https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/my-detection.py, https://github.com/dusty-nv/jetson-inference/blob/master/docs/imagenet-example-python-2.md

[cczarnuch]

Currently on the Jetson Nano:

• The YOLOv5 detections are taking the current time as follows: -CPU: 2.5-2.7 seconds per image -GPU: 0.27-0.54 seconds per image

The CPU does take a lot longer per image so we definitely will not use that for running the models. However, the GPU is very inconsistent in my set of test images (when compared to the cpu). This may be an optimization issue with the container as there is currently a lot of things running inside of is which are not going to be there in the final version.

Also, the model takes a very long time to load. After it has loaded the detections are very quick but initially there is a long waiting period. This may be an SD card bottleneck. I'll have to look into this more.

Lastly, one thing to note is that we are running the Jetson Nano in low-power mode because I don't have access to a better power supply. Let's look into getting one so that we can run at full-power.

[cczarnuch]

Just for some comparison, here is a screenshot from the jetson nano in a container: