This ROS package makes it simple to use CSI cameras on the Nvidia Jetson TK1, TX1, or TX2 with ROS via gstreamer and the Nvidia multimedia API. This is done by properly configuring gscam
to work with the Nvidia hardware.
Features
In order to get started, you will need to download the jetson_csi_cam
repository as well as its dependencies. Just follow the steps below and you should be good to go.
If you'd like to learn more about gscam
, check out their ROS wiki page or their Github repository.
Note: This package was tested on a Nvidia Jetson TX2 with L4T R27.1, ROS Kinetic, and the Leopard Imaging IMX377CS CSI camera.
For the purpose of this guide, we will assume you already have:
gscam
that supports gstreamer-1.0 is available for that ROS version, but this is untested.gscam
with gstreamer-1.0 support.
gscam
from source to support this, so don't worry about it yet.With these dependencies accounted for, lets get everything installed.
jetson_csi_cam
Clone this repository into you catkin_workspace
.
cd ~/catkin_workspace/src
git clone https://github.com/peter-moran/jetson_csi_cam.git
gscam
with gstreamer-1.0 supportClone gscam
into your catkin_workspace
.
cd ~/catkin_workspace/src
git clone https://github.com/ros-drivers/gscam.git
Then edit ./gscam/Makefile
and add the CMake flag -DGSTREAMER_VERSION_1_x=On
to the first line of the file, so that it reads:
EXTRA_CMAKE_FLAGS = -DUSE_ROSBUILD:BOOL=1 -DGSTREAMER_VERSION_1_x=On
While this flag is only necessary if you have both gstreamer-0.1
and gstreamer-1.0
installed simultaneously, it is good practice to include.
Now we build and register gscam
and jetson_csi_cam
in ROS.
cd ~/catkin_workspace
catkin_make
source ~/.bashrc
At this point everything should be ready to go.
TL;DR: To publish the camera stream to the ROS topic
/csi_cam_0/image_raw
, use this command in the terminal:roslaunch jetson_csi_cam jetson_csi_cam.launch width:=<image width> height:=<image height> fps:=<desired framerate>
If you have another camera on your Jetson TX2, to publish the other camera stream to the ROS topic
/csi_cam_1/image_raw
, use this command in the terminal:roslaunch jetson_csi_cam jetson_csi_cam.launch sensor_id:=1 width:=<image width> height:=<image height> fps:=<desired framerate>
If you would like to learn more of the details, read ahead.
To publish your camera's video to ROS (using the default settings) execute the following:
roslaunch jetson_csi_cam jetson_csi_cam.launch
Wait, where is the video? This launch file only publishes the video to ROS, making it available for other programs to use. This is because we don't want to view the video every time we use the camera (eg the computer may be processing it first). Thus we use separate programs to view it. I'll discuss this in a later section, but if you can't wait, run
rqt_image_view
in a new terminal to see the video.
You can confirm the video is running by entering rostopic list
in the terminal. You should be able to see the /csi_cam/image_raw
topic (aka your video) along with a bunch of other topics with similar names -- unless you changed the camera_name
argument from the default.
Most of the time we'll want to use settings other than the defaults. We can easily change these by passing command line arguments to roslaunch
. For example, if I want the camera to run at 4k resolution at 15 fps, I would use the following:
roslaunch jetson_csi_cam jetson_csi_cam.launch width:=3840 height:=2160 fps:=15
In other words, to set any of the arguments use the <arg_name>:=<arg_value>
options for roslaunch
.
jetson_csi_cam.launch
sensor_id
-- The sensor id of each camerawidth
-- Image Widthheight
-- Image Heightfps
-- Desired framerate. True framerate may not reach this if set too high.cam_name
-- The name of the camera (corrsponding to the camera info).frame_id
-- The TF frame ID for the camera.sync_sink
-- Whether to synchronize the app sink. Setting this to false may resolve problems with sub-par framerates.To view the video, simply run rqt_image_view
in a new terminal. A window will pop up and the video should be inside. You may need to go to the pulldown in the top left to choose your camera's video topic, by default you want to use /csi_cam/image_raw
.
To check the true framerate of your video you can use the rostopic hz
tool, which shows the frequency any topic is published at.
rostopic hz /csi_cam/image_raw
When the true framerate is below the framerate you asked for, it is either because the system cannot keep up or the camera does not have that setting. If you are using the Nvidia Jetson TX2, you can get higher resolution video at higher true FPS by switching to a higher power mode. It can also be good to dial back your requested FPS to be close or slightly above to your true FPS.
The jetson_csi_cam
package is set up to make camera calibration very simple. To calibrate your camera, all you need to do is follow the monocular camera calibration guide on the ROS wiki, with the following notes:
As the guide states, you'll need a printout of a chessboard. If you want something quick, use this chessboard for 8.5"x11" paper with an 8x6 grid. Please note: while there are nine by seven squares, we use a size of 8x6 because we are counting the internal vertices. You will need to measure the square size yourself since printing will slightly distort the chessboard size, but the squares should be 30mm on each side.
In Step 2, make sure to start the camera via roslaunch
as discussed above.
In Step 3, make sure to set your image
and camera
arguments correctly as below, also make sure that the chessboard size and square size are set correctly for your chessboard.
rosrun camera_calibration cameracalibrator.py --size 8x6 --square <square size in meters> image:=/csi_cam/image_raw camera:=/csi_cam
After following this guide and clicking COMMIT (as it tells you to do), your calibration data should be automatically published with your video under the topic /csi_cam/camera_info
. Most other ROS packages needing this information will automatically recognize it.