search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.52k stars 4.81k forks source link

[Intel RealSense D435 Python API, Ubuntu 22.04] frames = pipeline.wait_for_frames() RuntimeError: Frame didn't arrive within 5000 #11973

Closed monajalal closed 1 year ago

monajalal commented 1 year ago

Hi @MartyG-RealSense thanks a lot for all your supports in the issues. I have no problem running this code with my laptop at the office but we use a USB extender over there (which I don't have with me now). When I directly connect the USB cable to either of USB ports on my machine, I get the following error for the code below:

(calib) mona@ard-gpu-01:~/intel_realsense_captures$ cat capture.py 
import pyrealsense2 as rs
import numpy as np
import cv2

pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

color_path = 'V00P00A00C00_rgb.avi'
depth_path = 'V00P00A00C00_depth.avi'
colorwriter = cv2.VideoWriter(color_path, cv2.VideoWriter_fourcc(*'XVID'), 30, (640,480), 1)
depthwriter = cv2.VideoWriter(depth_path, cv2.VideoWriter_fourcc(*'XVID'), 30, (640,480), 1)

pipeline.start(config)
profile = pipeline.get_active_profile()
color_intrinsics = profile.get_stream(rs.stream.color).as_video_stream_profile().get_intrinsics()

try:
    while True:
        frames = pipeline.wait_for_frames()
        depth_frame = frames.get_depth_frame()
        color_frame = frames.get_color_frame()
        if not depth_frame or not color_frame:
            continue

        #convert images to numpy arrays
        depth_image = np.asanyarray(depth_frame.get_data())
        color_image = np.asanyarray(color_frame.get_data())
        depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth_image, alpha=0.03), cv2.COLORMAP_JET)

        colorwriter.write(color_image)
        depthwriter.write(depth_colormap)

        cv2.imshow('Stream', depth_colormap)

        if cv2.waitKey(1) == ord("q"):
            break
finally:
    colorwriter.release()
    depthwriter.release()

        # Print the intrinsic parameters
    print("Color Intrinsics:")
    print("Width:", color_intrinsics.width)
    print("Height:", color_intrinsics.height)
    print("Principal Point (x, y):", color_intrinsics.ppx, color_intrinsics.ppy)
    print("Focal Length (x, y):", color_intrinsics.fx, color_intrinsics.fy)

    pipeline.stop()

as you see it shows the first frame and then throws the following error:

(calib) mona@ard-gpu-01:~/intel_realsense_captures$ python capture.py 
Color Intrinsics:
Width: 640
Height: 480
Principal Point (x, y): 321.5348815917969 235.87950134277344
Focal Length (x, y): 616.056884765625 616.2793579101562
Traceback (most recent call last):
  File "/home/mona/intel_realsense_captures/capture.py", line 22, in <module>
    frames = pipeline.wait_for_frames()
RuntimeError: Frame didn't arrive within 5000

Screenshot from 2023-07-06 15-02-50

It also creates the video out. Screenshot from 2023-07-06 15-05-19

I also have problem with realsense-viewer: Screenshot from 2023-07-06 15-06-03

$ uname -a
Linux ard-gpu-01 5.19.0-46-generic #47~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jun 21 15:35:31 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
LSB Version:    core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:    22.04
Codename:   jammy

$ lsusb
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 013: ID 8086:0b07 Intel Corp. RealSense D435
Bus 003 Device 003: ID 04f2:b6be Chicony Electronics Co., Ltd Integrated Camera
Bus 003 Device 010: ID 06cb:00bd Synaptics, Inc. Prometheus MIS Touch Fingerprint Reader
Bus 003 Device 005: ID 8087:0032 Intel Corp. AX210 Bluetooth
Bus 003 Device 012: ID 17ef:608d Lenovo Optical Mouse
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

$ sudo dmidecode -t system | grep "Product Name"
[sudo] password for mona: 
    Product Name: 20YS004SUS  --> Lenovo Thinkpad with mobile 3080 GPU 

$ sudo lshw -short | grep system
USB                         
                                 system         20YS004SUS (LENOVO_MT_20YS_BU_Think_FM_ThinkPad T15g Gen 2i)

I am also getting this messages:

$ sudo dmesg
[25827.629849] gigedrv: gige_hf_register called
[25845.527761] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25845.972124] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.022978] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.074045] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.125262] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.177959] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.228912] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.280062] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.331225] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.484449] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.535618] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.689036] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25846.740089] usb 3-7: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[25853.274806] gigedrv: gige_hf_unregister called
[25856.714869] gigedrv: gige_hf_unregister called
[25880.417064] gigedrv: gige_hf_register called

also:

$ sudo usb-devices

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480 MxCh= 1
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev=05.19
S:  Manufacturer=Linux 5.19.0-46-generic xhci-hcd
S:  Product=xHCI Host Controller
S:  SerialNumber=0000:00:0d.0
C:  #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

T:  Bus=02 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=10000 MxCh= 4
D:  Ver= 3.10 Cls=09(hub  ) Sub=00 Prot=03 MxPS= 9 #Cfgs=  1
P:  Vendor=1d6b ProdID=0003 Rev=05.19
S:  Manufacturer=Linux 5.19.0-46-generic xhci-hcd
S:  Product=xHCI Host Controller
S:  SerialNumber=0000:00:0d.0
C:  #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

T:  Bus=03 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480 MxCh=16
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev=05.19
S:  Manufacturer=Linux 5.19.0-46-generic xhci-hcd
S:  Product=xHCI Host Controller
S:  SerialNumber=0000:00:14.0
C:  #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

T:  Bus=03 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 12 Spd=1.5 MxCh= 0
D:  Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs=  1
P:  Vendor=17ef ProdID=608d Rev=01.00
S:  Manufacturer=PixArt
S:  Product=lenovo USB Optical Mouse
C:  #Ifs= 1 Cfg#= 1 Atr=a0 MxPwr=100mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=03(HID  ) Sub=01 Prot=02 Driver=usbhid
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=10ms

T:  Bus=04 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=20000 MxCh= 6
D:  Ver= 3.10 Cls=09(hub  ) Sub=00 Prot=03 MxPS= 9 #Cfgs=  1
P:  Vendor=1d6b ProdID=0003 Rev=05.19
S:  Manufacturer=Linux 5.19.0-46-generic xhci-hcd
S:  Product=xHCI Host Controller
S:  SerialNumber=0000:00:14.0
C:  #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

I am using an Intel RealSense D435 made in Thailand (silver).

Please suggest what could be the problem.

From https://github.com/IntelRealSense/librealsense/issues/6628 I assume this is a USB issue but I am not entirely sure what the solution is.

Required Info
Camera Model { R200 / F200 / SR300 / ZR300 / D400 }
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version {Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS
Kernel Version (Linux Only) (e.g. 4.14.13)
Platform PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version { legacy / 2.<?>.<?> }
Language {C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment {Robot/Smartphone/VR/AR/others }

Issue Description

<Describe your issue / question / feature request / etc..>

MartyG-RealSense commented 1 year ago

You are very welcome, @monajalal :)

How long is your extender cable, please? If it is longer than 2 meters then it is important to use a high quality cable that is designed for data transfer because of the high volume of data that RealSense cameras can transmit.

If the cable is longer than 5 meters then I would recommend one with active repeater components in it to boost the signal over distance. The company Newnex supplies high quality USB cables that have been validated for use with RealSense.

https://www.newnex.com/realsense-3d-camera-connectivity.php

monajalal commented 1 year ago

Thanks for your response. I am using the same exact camera and cable that I used in the office. I just forgot to bring the USB extender we connect for our mouse and cameras.

I am curious if the USB ports on Thinkpad laptop may cause some problem? The cable is as shown in the photos and is a bit over 1m. I will check again when I am back in the office on Monday and update you and share more details about the USB extender I've been using.

MicrosoftTeams-image (33) MicrosoftTeams-image (34)

MartyG-RealSense commented 1 year ago

Thank you very much for the details and images.

I use a ThinkPad like yours with RealSense cameras extensively myself. It is compatible with all RealSense models, though I find that some specific RealSense camera units may disconnect, reconnect and drop the connection again. Using the single USB port on the side of the computer with the ventilation grille (by the Delete key) can improve USB performance of affected camera units on my particular computer.

monajalal commented 1 year ago

I'm gonna close this issue since when I use this usb connector, I don't have the problem MicrosoftTeams-image (35)

MartyG-RealSense commented 1 year ago

It's great to hear that you found a solution, @monajalal - thanks very much for the update!

animesh-singhal commented 7 months ago

@monajalal: Thanks! A USB Hub also fixed my issue. @MartyG-RealSense: I use AMD Ryzen 9, so the processor isn't the problem.

MartyG-RealSense commented 7 months ago

Hi @animesh-singhal Thanks very much for the confirmation that a USB hub worked for you too!