search

IvLabs / autonomous-delivery-robot

Repository for Autonomous Delivery Robot project of IvLabs, VNIT
http://www.ivlabs.in/autonomous-driving-platform.html
68 stars 18 forks source link

Semantic Segmentation testing on Jetson TX1 #1

Closed akshaykulkarni07 closed 4 years ago

akshaykulkarni07 commented 4 years ago

We need to test the semantic segmentation model (of Keras/TF) on Nvidia Jetson TX1, calculate the FPS and identify whether we require additional hardware for this computation.

akshaykulkarni07 commented 4 years ago

This task requires installation of TensorFlow on Jetson. Currently, there is some issue with the setup of Jetson (the OS is not on SD card), so enough space is not available for TF installation. Workaround is provided here.

Jetson Zoo provides useful installation procedures for common softwares.

akshaykulkarni07 commented 4 years ago

The installation of Jetpack 3.1 has CUDA 8.0, while TensorFlow requires CUDA 9.0. So, we tried the installation of both Jetpack 3.3 and 3.2 which has CUDA 9.0, but there were some issues during the installation, so it needs to be repeated. This will be done after coming back from home (have to work on FYP presentation now).

akshaykulkarni07 commented 4 years ago

Take a look at testing videos processed on the GTX 1080Ti at this link.

akshaykulkarni07 commented 4 years ago

Shifted 18.04 Ubuntu to SD Card (using same link mentioned above). Installed PyTorch 1.3.0 using this link for Python 2 because ROS supports only Python2. torchvision library is installed using this link.

Will shift all code to PyTorch (and Python2) now. This will take at least one week.

akshaykulkarni07 commented 4 years ago

Shifted all code (of ENet) to PyTorch (Python2). Tested the code on Jetson TX1, got around 8 FPS for inputs of 640x320. However, network isn't trained fully yet. Need to do this, but the code works on Jetson, so this task is done.

Note: We had to add a swap file (of 8 GB) since Jetson TX1 has a shared RAM and VRAM of total 4 GB. Otherwise, a low memory warning and subsequent killing of process occurs on running the model.

navidpanchi commented 4 years ago

You have predicted on a single image at a time, you should use batch prediction. That could give many more fps than this.

akshaykulkarni07 commented 4 years ago

What do we do with those multiple predictions? I think we need only the latest one. Unless we are thinking of averaging the predictions of the last few images. It's probably not a good idea because the scene will have changed. Also, even if we do batch prediction, it will still require at least the time for a single image (0.125 seconds according to 8 FPS). And we won't be predicting images continuously anyway, so there's no advantage in batch processing.
Any further comments @navidpanchi?