Use CSI camera source

[mrtj]

I am trying to configure a CSI camera attached directly to the Jetson board to work with your demo. I set up the app config file as follows:

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP 5=CSI
type=5
camera-width=1280
camera-height=720
camera-fps-n=30
camera-fps-d=1
camera-csi-sensor-id=0

This config file works perfectly if I run the deepstream-app manually. However if I try to run it from greengrass, the gstreamer pipeline construction fails with this error:

[2020-07-23T19:35:42.165+02:00][ERROR]-ERROR from src_elem: Cannot identify device '/dev/video0'.
[2020-07-23T19:35:42.165+02:00][ERROR]-Debug info: v4l2_calls.c(609): gst_v4l2_open (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
[2020-07-23T19:35:42.165+02:00][ERROR]-system error: No such file or directory
[2020-07-23T19:35:43.141+02:00][INFO]-run_deepstream.py:46,('\n *** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***\n\nOpening in BLOCKING MODE \n** ERROR: <m
ain:651>: Failed to set pipeline to PAUSED\nQuitting\nApp run failed\n', None)

So I added /dev/video0 as a device type local resource to the greengrass group. The next error was:

[2020-07-23T20:30:28.208+02:00][ERROR]-(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch
.cpp, function openSocketConnection(), line 201)
[2020-07-23T20:30:28.208+02:00][ERROR]-(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvid
er(), line 102)
[2020-07-23T20:30:28.208+02:00][ERROR]-Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:551 Failed to create 
CameraProvider

This seemed to be more tricky. It turns out that nvarguscamerasrc plugin has to have access to two special file descriptors as well, /tmp/argus_socket and /tmp/.X11-unix. This later is a directory-like file descriptor so I can map it as a file resource in the gg group but the argus_socket one is a file-like descriptor that I can't map directly as my gg deploy fails with an error saying "the specified volume is not a valid directory nor regular file" (that in fact is true). Now I found a turnaround to map the whole /tmp directory directly in the gg container as well, and the camera source works as expected. However I don't think it is a good idea exposing the whole /tmp to the lambda as it clearly violates some security principles like that one of the least privilege necessary. Do I have some other way to make /tmp/argus_socket visible in the container?

[yuxiny2586]

Sorry, I just saw your comment. I will watch it more closely at this point.

Have you tried to attach this in as a volume resource? https://docs.aws.amazon.com/greengrass/latest/developerguide/lra-console.html

Let me try it really quick.

[mrtj]

Yes, if I mount it as a local resource the deployment fails with the above error: "the specified volume is not a valid directory nor regular file"

[yuxiny2586]

Could you try mounting the entire /tmp folder as a volume? I have tried that once and it at least it can mount successfully (have not tried with CSI camera yet).

[AndreV84]

@mrtj you may try emulating USB camera[v4l2] from CSI camera then see if it worked as with USB camera using CSI camera [in case of jetpack 4.4.1]

sudo su
cd /usr/src/linux-headers-4.9.140-tegra-ubuntu18.04_aarch64/kernel-4.9
## dropped in latter release##make modules_prepare
mkdir v4l2loopback
git clone https://github.com/umlaeute/v4l2loopback.git v4l2loopback
cd v4l2loopback && git checkout -b v0.10.0
make
make install
apt-get install -y v4l2loopback-dkms v4l2loopback-utils
modprobe v4l2loopback devices=1 video_nr=2 exclusive_caps=1

that will create a temporary v4l2loopback that could be activated with:

gst-launch-1.0 -v nvarguscamerasrc ! 'video/x-raw(memory:NVMM), format=NV12, width=1920, height=1080, framerate=30/1' ! nvvidconv ! 'video/x-raw, width=640, height=480, format=I420, framerate=30/1' ! videoconvert ! identity drop-allocation=1 ! 'video/x-raw, width=640, height=480, format=RGB, framerate=30/1' ! v4l2sink device=/dev/video2

so from this point the CSI camera works as USB camera so you could read from it as from V4l2 source [in case of jetpack 4.5] the first part slightly differ with one line difference

cd /usr/src/linux-headers-4.9.201-tegra-ubuntu18.04_aarch64/kernel-4.9/

[AndreV84]

@mrtj

src_elem: Cannot identify device '/dev/video0'.

are you able to see the device /dev/video0 with the command below?

ls /dev/video*

are you able to run on jetson, as you are you are using a jetson device, do not you? could you see the camera outputs with executing the command below?

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=120/1, format=NV12' ! nvvidconv flip-method=0 ! nvegltransform ! nveglglessink -e

[yuxiny2586]

If you have trouble with the GG deployment, could you paste your GG lambda logs here for us to take a look? Maybe there is some error messages within that log file that gives us clue on what component we are missing.