Storage on the eMMC Module

[tawfiq1200]

@n-snyder @murphym18

I think we need to know what is the expected amount of free storage after the base image has been put on the Jetson.

There are currently different amounts of storage in the two Jetsons on the two drones. This image shows the storage on the first drone after I uninstalled any libraries or git cloned anything.

This is the storage on the Jetson-01 on the first/red drone:

This is the apt list of all the installed applications on that drone: apt.txt

This is the storage on the Jetson-02 on the second/blue drone:

So, I am working on the red drone first to install CUDA and CUDNN. This tells that CUDA takes about 2.2 GB space and CUDNN takes about 300 MB space for installation on Windows and I think for Ubuntu, it is about the same. Yesterday, when I was following this link to try to install CUDA, I had to abort the installation as there was a possibility of the eMMC module running out of storage the and the Jetson crashing. The file system won't show anything so I was not having a clue.

Then later I remembered that we had a similar problem on our sarec2 server where we git pulled the latest code and kept building new docker images and they keep taking memories on the background. This is something we need to be careful about. This screenshot I took this morning shows the currently available docker images on the red drone, most of them are obsolete. We may need to tag the images we want to keep. Otherwise, if we keep updating the code and keep building new docker images, at some point, the eMMC module will crash.

I will regularly clean images from now.

[n-snyder]

@tawfiq1200, thanks for showing this. On Jetson-02, I did a clean build of the image using all the lessons learned from building Jetson-01. After that, I made a copy of the image as my source for cloning the next batch of aircraft.

By the way, I used the Docker application of SDK Manager to initially create the image before updating the image with the appropriate device tree, services, etc. This page highlights some of the additional components that you can select in SDK Manager, including CUDA. I selected components based on the components needed for the hardware-tests. If there are additional components that you will need for flight ops, we can eventually update our processes to accommodate your requirements. Remember that space is limited.

[tawfiq1200]

After a clean installation of all the code, it appears that there are 4.1 GB space available in both the Jetsons. I may tag the images from now on to keep track of changes and run sudo docker system prune to remove all the unused containers.

@n-snyder Thank you for sharing that link. It seems that packages need to be installed at the time of burning the image to eMMC. If it is not true, please let us know. We need openCV, CUDA and CUDNN for our computer vision pipeline so you may consider putting them on the images for other drones. We will follow instructions from here when the packages are installed. These instructions are for YOLO V3 and V4.

[tawfiq1200]

@n-snyder @murphym18

This link that I shared in my previous comment is for Jetsons with GPU. When it says GPU, I think it refers to a dedicated GPU. The Jetson nano developers kit comes with a 128-core Maxwell dedicated GPU and Jetson NX developer's kit comes with a NVIDIA Volta architecture with 384 NVIDIA CUDA® cores and 48 Tensor cores. They both have dedicated GPUs. However, it seems that the carrier board on the drone does not have a dedicated GPU. Please look at this screenshot:

Every computer must have at least an integrated GPU. For instance, the basic Windows laptops come with Intel HD Graphics but the gaming ones come with AMD or NVidia GPUs.

So, the question is does that integrated GPU count as a GPU or it is part of the CPU?

If it does not have a GPU, then there is no point installing CUDA and CUDNN. We should however still be able to use OpenCV but in that case, all the processing will be done by the CPU alone including the general application as well as the computer vision.

[tawfiq1200]

It says that we can compile CUDA without a dedicated GPU but I am not sure if it will actually serve the purpose of CUDA.

[tawfiq1200]

We probably should go ahead with CUDA even with an integrated GPU based on this, I looked at the "Application Notes".

[n-snyder]

@tawfiq1200, the Xavier NX module has a 384-core Maxwell GPU. This is true whether the module is installed on the dev kit's carrier board or on another carrier board.

[tawfiq1200]

@n-snyder

This is great to know. That means the Nvidia drivers for that GPU is not installed which is why the system does not detect it. The GPU is as good as a nano's dedicated GPU but it is still good to have one.

[tawfiq1200]

Sorry, it has the standard one for NX, not the nano but drivers need to be installed.

[tawfiq1200]

@n-snyder

I have found this documentation on the Nvidia website which gives us a proper way of installing the required packages on an existing image using this command:

sudo apt install nvidia-jetpack

I started the installation on the first/red drone and at the beginning, there was 4.4 GB of space available. Then at some point of the installation process, these messages appeared:

I removed all docker images as the storage completely ran out of space so we cannot fly the drone with code now. After that cleanup, it now has 2 GB of free space. It went to the point that it won't allow me taking screenshots and I had to use my phone to capture photos.

Checking in the CUDA directory, I do find an installation of CUDA so that part seems to have been completed.

I however, did not find the dedicated 384-core Maxwell GPU to show up in the Settings (About). It looks like though that the standard Jetson NX developer's kit has a 6 core CPU but the one on the carrier board has 4 cores. This screenshot is from August 12, 2022:

So, I am not actually sure that they both have the same hardware in terms of GPU and the CPU.

However, the key point is that these drones will be used for the new few years at least I believe and they are meant to use computer vision onboard. The Ubuntu as well as other installed packages will receive updates over the years and 14 GB storage in 2022 does not seem enough to me when you need to install everything on that storage size. We have used the developer's kit with SD cards as high as 512 GB. We need at least 32 GB for it to function properly.

I think extending the size of the eMMC is now a priority.

[tawfiq1200]

@n-snyder

Actually I think we should go with at least 64 GB.

[n-snyder]

@tawfiq1200, the information you provided is helpful to scope the effort. If disk space is pressing right now, have you considered moving the docker repository and larger apps to the SD card?