jetson TX2 camera

[imxboards]

hi , I install tx2 with Ubuntu 16.04 Python 2.7 Tensorflow 1.7 OpenCV 3.3.1 But when I ruan object_detection.py issue is coming

vidia@tegra-ubuntu:~/tf/realtime_object_detection$ python object_detection.py

Model found. Proceed. Loading frozen model into memory 2018-03-30 07:11:08.517989: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:865] ARM64 does not support NUMA - returning NUMA node zero 2018-03-30 07:11:08.518123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties: name: NVIDIA Tegra X2 major: 6 minor: 2 memoryClockRate(GHz): 1.3005 pciBusID: 0000:00:00.0 totalMemory: 7.66GiB freeMemory: 2.29GiB 2018-03-30 07:11:08.518172: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0 2018-03-30 07:11:10.014011: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix: 2018-03-30 07:11:10.014083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0 2018-03-30 07:11:10.014109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N 2018-03-30 07:11:10.014269: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1694 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2) Loading label map Building Graph 2018-03-30 07:11:33.197609: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0 2018-03-30 07:11:33.197707: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix: 2018-03-30 07:11:33.197737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0 2018-03-30 07:11:33.197758: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N 2018-03-30 07:11:33.197847: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1694 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2) start!!! 2018-03-30 07:11:33.199546: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0 2018-03-30 07:11:33.199636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix: 2018-03-30 07:11:33.199671: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0 2018-03-30 07:11:33.199713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N 2018-03-30 07:11:33.199817: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1694 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2) 2018-03-30 07:11:33.200175: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0 2018-03-30 07:11:33.200259: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix: 2018-03-30 07:11:33.200292: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0 2018-03-30 07:11:33.200315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N 2018-03-30 07:11:33.200394: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1694 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2) VIDEOIO ERROR: V4L2: Pixel format of incoming image is unsupported by OpenCV OpenCV Error: Unspecified error (GStreamer: unable to start pipeline ) in cvCaptureFromCAM_GStreamer, file /home/nvidia/opencv/modules/videoio/src/cap_gstreamer.cpp, line 887 VIDEOIO(cvCreateCapture_GStreamer(CV_CAP_GSTREAMER_V4L2, reinterpret_cast<char *>(index))): raised OpenCV exception:

/home/nvidia/opencv/modules/videoio/src/cap_gstreamer.cpp:887: error: (-2) GStreamer: unable to start pipeline in function cvCaptureFromCAM_GStreamer

Start video stream with shape: 0,0 Press 'q' to Exit Starting Detection OpenCV Error: Assertion failed (scn == 3 || scn == 4) in cvtColor, file /home/nvidia/opencv/modules/imgproc/src/color.cpp, line 11016 Traceback (most recent call last): File "object_detection.py", line 302, in main() File "object_detection.py", line 298, in main detection(graph, category, score, expand) File "object_detection.py", line 213, in detection image_expanded = np.expand_dims(cv2.cvtColor(image, cv2.COLOR_BGR2RGB), axis=0) cv2.error: /home/nvidia/opencv/modules/imgproc/src/color.cpp:11016: error: (-215) scn == 3 || scn == 4 in function cvtColor

can you help me ? should i change config.yml parm?

[gustavz]

are you using the onboard camera of the jetson? do you know how to use it in opencv, have a look here: https://devtalk.nvidia.com/default/topic/1017860/using-on-board-tx2-camera-with-opencv-3-2-/?offset=7 and others

[Alro10]

Hi! for my experience: In config.sample.yml

video_input : nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)600, height=(int)600,format=(string)I420, framerate=(fraction)5/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink

[warisz]

@AlexanderRobles21 @GustavZ Hi, I tried setting video_input as you suggested for Jetson TX2 camera, however, I get the following error.

nvidia@tegra-ubuntu:~/Desktop/realtime_object_detection$ python object_detection.py 
> Model found. Proceed.
> Loading frozen model into memory
2018-04-26 22:56:07.466932: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:865] ARM64 does not support NUMA - returning NUMA node zero
2018-04-26 22:56:07.467125: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties: 
name: NVIDIA Tegra X2 major: 6 minor: 2 memoryClockRate(GHz): 1.3005
pciBusID: 0000:00:00.0
totalMemory: 7.66GiB freeMemory: 2.29GiB
2018-04-26 22:56:07.467194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-04-26 22:56:10.025344: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1687 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
> Loading label map
> Building Graph
2018-04-26 22:56:32.894441: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-04-26 22:56:32.894590: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 577 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)600, height=(int)600,format=(string)I420, framerate=(fraction)5/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink: Unable to query number of channels
2018-04-26 22:56:32.897282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-04-26 22:56:32.897409: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 577 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
2018-04-26 22:56:32.902302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-04-26 22:56:32.902475: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 577 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)

Available Sensor modes : 
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

Socket read error. Camera Daemon stopped functioning.....
gst_nvcamera_open() failed ret=0
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in icvStartPipeline, file /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp, line 450
Traceback (most recent call last):
  File "object_detection.py", line 301, in <module>
    main()
  File "object_detection.py", line 297, in main
    detection(graph, category, score, expand)
  File "object_detection.py", line 201, in detection
    video_stream = WebcamVideoStream(video_input,width,height).start()
  File "/home/nvidia/Desktop/realtime_object_detection/stuff/helper.py", line 108, in __init__
    self.stream.set(cv2.CAP_PROP_FRAME_WIDTH, self.width)
cv2.error: /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp:450: error: (-2) GStreamer: unable to start pipeline
 in function icvStartPipeline

I have installed applicable gstreamer dev packages. I am able to invoke the camera through terminal via gst-launch, for example gst-launch-1.0 -ev nvcamerasrc ! nvvidconv ! ximagesink and can also access camera via other scripts for example

  # Use Jetson onboard camera
    gst_str = ("nvcamerasrc ! "
               "video/x-raw(memory:NVMM), width=(int)2592, height=(int)1458, format=(string)I420, framerate=(fraction)30/1 ! "
               "nvvidconv ! video/x-raw, width=(int){}, height=(int){}, format=(string)BGRx ! "
               "videoconvert ! appsink").format(width, height)
    return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)

Any help would be appreciated. Thanks.

[gustavz]

I make a guess: you are using OpenCV that comes preinstalled with flashing JetPack right?

The problem is that Nvidia/JetPack Build OpenCV without Gstreamer support. That’s why it is not possible to use the onboard camera with the standard OpenCV (ask NVIDIA why they install OpenCV that way that it can’t use its own hardware. They also disabled CUDA support for OpenCV, so i guess they dont want you to use it but instead Visionworks...)

Long story short: you need to completely deinstall OpenCV and reinstall it from source with Gstreamer Support Enabled.

[warisz]

Hi Gustav,

Thanks for your quick reply. I actually reinstalled OpenCV from source following this tutorial: https://jkjung-avt.github.io/opencv3-on-tx2/ I am able to launch the onboard camera, like for example, via the last code snippet I posted in my previous message.

I am using TF1.6 with JetPack 3.1 (and have also tried TF1.5 and gotten same error). From prying around the other issues in this repo, TF1.4 is recommended for this repo; however the error I have posted seems to be Gstreamer related.

I will try reinstalling OpenCV from source and switch to JetPack3.1 to use TF1.4 and try that and post an update. Any input from your side is also appreciated.

Thanks a lot.

[gustavz]

I use and always have used JetPack 3.2, maybe you should switch to that?

[Alro10]

Hi @imxboards,

I recently tested object_detection.py for Jetpack 3.1 and TF 1.3 without any problems using onboard cam and webcam ( video_input = /dev/video1) achieved 20fps. But when I used TF 1.6 rc1 there is a problem:

2018-04-27 11:06:05.483867: E tensorflow/stream_executor/event.cc:33] error destroying CUDA event in context 0x337fbf0: CUDA_ERROR_LAUNCH_FAILED

I recommend to use TF 1.3, I think this version works for CUDA 8. For Jetpack 3.2 (CUDA 9) I didn't tested yet.

[gustavz]

I successfully tested it on 1.4, 1.5, 1.6 but with varying performance results.

If you want to achieve highest results use the versions that I recommend in the readme.

Also I strongly recommend you all to use JetPack 3.2 because of CUDA 9 / cuDNN 7 among other improvements

[lfelipesv]

@warisz could you solve the problem? I am experiencing the same problem trying to install with tensorflow 1.6 and opencv 3.4.0 that i built by myself... cheers!

[scbean]

I am experiencing the same problem, I have Jetpack 3.3 and TF 1.9

I installed open CV using this tutorial : https://jkjung-avt.github.io/opencv3-on-tx2/

VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink: Unable to query number of channels

Available Sensor modes : 2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10 2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10 1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

Available Sensor modes : 2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10 2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10 1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

Socket read error. Camera Daemon stopped functioning..... OpenCV Error: Unspecified error (GStreamer: unable to start pipeline ) in icvStartPipeline, file /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp, line 450 Traceback (most recent call last): File "run_stream.py", line 123, in main detection.start(cfg) File "/home/nvidia/Desktop/realtime_object_detection-master/lib/detection_nms_v2.py", line 192, in start video_reader.start(VIDEO_INPUT, WIDTH, HEIGHT, save_to_file=SAVE_TO_FILE) File "/home/nvidia/Desktop/realtime_object_detection-master/lib/webcam.py", line 45, in start self.vid.set(cv2.CAP_PROP_FRAME_HEIGHT, height) cv2.error: /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp:450: error: (-2) GStreamer: unable to start pipeline in function icvStartPipeline

[Merwanski]

Hi @scbean have similar problem to the one you are describing did you find a solution for it ???

Merwan

[Alro10]

Hi @Merwanski and @scbean

You can check my repository: https://github.com/Alro10/realtime_object_detection

Use this file: object_detectionjetson.py

Please write back if that solves your problem!

[Merwanski]

Thank you @Alro10 ... the camera is working but I am still getting this error message """""" VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink: Unable to query number of channels """""""