The NVIDIA Jetson platform supports cloud-native technologies and workflows such as containerization and orchestration. This support enables application development, deployment and management at scale, which is essential to deliver AI at the edge to millions of devices.
This demo is built around example AI applications for a service robot use case. It was created specifically to showcase the capabilities of Jetson Xavier NX. (The complete demo will also run on Jetson AGX Xavier, but not other Jetsons, as some parts leverage Tensor cores not present in other Jetsons.)
Service robots are autonomous robots that need to interact with people in retail, hospitality, healthcare, warehouse, and other settings. For example, consider a customer service robot in a retail setting, interacting with customers and providing helpful answers to customer queries. Such a robot will need to perform the following tasks:
Hence the robot will need multiple AI models such as:
Following the cloud-native approach to application development, these individual models can be developed independently. Once an individual model is developed, it can be containerized with all dependencies included, and deployed to any Jetson device.
For this demo we have developed and containerized the models, which are hosted on NVIDIA NGC. This demo runs seven models simultaneously as described below:
These containers provide the building blocks of a service robot use case. Modifying applications and deploying updates is easy because of containerization. Other containers won't be affected by updates, giving zero down time and a seamless experience.
You would need to install JetPack 4.3 to run this demo on Jetson Xavier NX
To run the demo containers individually, refer to the corresponding instructions at each container's NGC page:
This demo requires two items in addition to a Jetson Xavier NX Developer Kit: 1) an M.2 NVMe drive, and 2) a USB Headset with microphone such as Logitech H110 or H390.
Since these demo containers are not yet fully optimized for storage and memory size, this demo requires NVMe for extra storage and adding swap space for extra virtual memory. The usual path for deploying containers into production involves optimization for size and memory usage. These demo containers have not yet gone through such optimizations.
The NVMe drive can be connected to the M.2 connector underneath the Jetson Xavier NX Developer Kit. Power down the developer kit and then connect the NVMe as shown in this picture:
Boot the developer kit, Format the NVMe, prepare the mount point, and then mount the NVMe. NOTE that these examples and following refer to /home/nvidia
, but you should replace with path to your home directory.
sudo mkfs.ext4 /dev/nvme0n1
sudo mkdir /home/nvidia/nvme
sudo mount /dev/nvme0n1 /home/nvidia/nvme
Once the NVMe is mounted, add the following to /etc/fstab and then reboot the developer kit. the NVMe storage will be automounted going forward.
/dev/nvme0n1 /home/nvidia/nvme ext4 defaults 0 1
Next, change the docker registry to point to NVMe so that the docker images are stored in NVMe. NOTE that the second command below is optional unless you have previously pulled some containers, in which case it is required to move those docker images to NVMe:
sudo mkdir /home/nvidia/nvme/docker
sudo mv /var/lib/docker/* /home/nvidia/nvme/docker/.
sudo ln -s /home/nvidia/nvme/docker /var/lib/docker
Next, create a file swap on NVMe by following these instructions:
Turn off zram:
cd /etc/systemd
sudo mv nvzramconfig.sh nvzramconfig.sh.orig
sudo reboot
Add swap file on nvme and verify:
sudo fallocate -l 32G /home/nvidia/nvme/swapfile
sudo chmod 600 /home/nvidia/nvme/swapfile
sudo mkswap /home/nvidia/nvme/swapfile
sudo swapon /home/nvidia/nvme/swapfile
sudo swapon -s
Add the line below to /etc/fstab so swap file will be automounted going forward:
/home/nvidia/nvme/swapfile swap swap defaults 0 0
Reboot the developer kit after saving the /etc/fstab changes.
Pull the 4 demo containers using the pull instruction mentioned in each container’s NGC page.
First clone this repository:
git clone https://github.com/NVIDIA-AI-IOT/jetson-cloudnative-demo
Install the xdotool app by running the command below:
sudo apt-get install xdotool
Go to the directory
cd jetson-cloudnative-demo
Launch the demo
sudo ./run_demo.sh
Top Left Quadrant - People Detection Container
The top left quadrant of the demo is running a containerized people detection inferencing task using NVIDIA DeepStream. It is analyzing four concurrent video streams to identify the number of people in each stream.
Top Right Quadrant - Natural Language Processing Container
The top right quadrant of the demo is running a containerized Natural Language Processing (NLP) demo using the demanding BERT NLP neural network. This demo takes your questions through voice input on specific topics and provides relevant answers based on the content available under each topic. Please follow these instructions to experience this part of the demo:
Bottom Left Quadrant - Pose Estimation
This container is running a pose estimation neural network to estimate the pose of people in the input video stream. For example, this information could be used by a retail service robot to figure out whether the person is pointing at a specific product in the store, or asking a delivery robot to stop based on the pose of the person’s hands, etc.
Bottom Right Quadrant - Gaze Estimation
This container is running a gaze estimation neural network to figure out whether the person in the frame is looking at the robot or looking somewhere else. Whenever the person looks at the robot, the boxes around the person’s eyes turn green. This information could be used to help the robot know when to interacting with the person.
To end the demo, please go back to the terminal window by clicking on the terminal icon on the left side of your screen and hit “Enter” to close all containers.