orbbec / OrbbecSDK

Orbbec SDK C/C++ base core lib
https://www.orbbec3d.com/
Other
97 stars 17 forks source link

Gemini 2 cannot reconnect over USB2.0 in HotPlugin example #100

Closed camkisailus closed 3 months ago

camkisailus commented 3 months ago

Hi! I am working with the Gemini 2 and trying to run the HotPlugin example. The program runs fine when the camera is plugged in to start. When I unplug and replug the camera, or start the program without a camera plugged in and then plug in a camera, the program can receive a few frames, but then the camera disconnects and ends up in a loop of disconnecting and reconnecting until I stop the program.

Here is sample output from a run where I started with a camera plugged in, unplugged it, and then reconnected it:

06/18 10:50:14.179594][info][17843][DeviceManager.cpp:15] Current found device(s): (1)
[06/18 10:50:14.179602][info][17843][DeviceManager.cpp:24]  - Name: Orbbec Gemini 2, PID: 0x0670, SN/ID: AY35C3200W2, Connection: USB2.0
Device connect, deviceCount: 1
[06/18 10:50:14.190848][info][17843][Gemini2Device.cpp:432] Device.OrbbecGemini2.FirmwareUpgradeFoolproof device firmware upgrade foolproof status true
[06/18 10:50:14.191772][info][17843][Gemini2Device.cpp:125] Gemini2 device created! PID:1648, SN:AY35C3200W2, depthMode: {name: Unbinned Dense Default, checksum(hex): 6bb28ca9a0dff5553497f9eecb59f667, optionCode: NORMAL}
[06/18 10:50:14.191792][info][17843][DeviceManager.cpp:154] Device created successfully! Name: Orbbec Gemini 2, PID: 0x0670, SN/ID: AY35C3200W2
[06/18 10:50:14.191825][info][17843][Pipeline.cpp:44] Pipeline created with device: {name: Orbbec Gemini 2, sn: AY35C3200W2}, @0x626287EF2CB0
Device connected. vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-55
startStream vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-55
[06/18 10:50:14.194099][info][17843][Gemini2Device.cpp:685] Color sensor has been created!
[06/18 10:50:14.199574][info][17843][Gemini2Device.cpp:637] Depth sensor has been created!
[06/18 10:50:14.201305][info][17843][Pipeline.cpp:238] Try to start streams!
[06/18 10:50:14.201334][info][17843][VideoSensor.cpp:590] start OB_SENSOR_COLOR stream with profile: {type: OB_STREAM_COLOR, format: OB_FORMAT_MJPG, width: 1280, height: 720, fps: 30}
[06/18 10:50:14.207548][info][17843][VideoSensor.cpp:590] start OB_SENSOR_DEPTH stream with profile: {type: OB_STREAM_DEPTH, format: OB_FORMAT_Y14, width: 320, height: 200, fps: 30}
[06/18 10:50:14.211498][info][17843][Pipeline.cpp:251] Start streams done!
[06/18 10:50:14.211507][info][17843][Pipeline.cpp:234] Pipeline start done!
=====Depth Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_DEPTH, index: 14, width: 320, height: 200, format: 8, timeStampUs: 146563271us, centerDepth: 284mm
=====Color Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_COLOR, index: 14, width: 1280, height: 720, format: 5, timeStampUs: 146645333us
=====Depth Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_DEPTH, index: 29, width: 320, height: 200, format: 8, timeStampUs: 147062907us, centerDepth: 284mm
=====Color Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_COLOR, index: 29, width: 1280, height: 720, format: 5, timeStampUs: 147145333us
=====Depth Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_DEPTH, index: 44, width: 320, height: 200, format: 8, timeStampUs: 147562543us, centerDepth: 284mm
=====Color Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_COLOR, index: 44, width: 1280, height: 720, format: 5, timeStampUs: 147645333us
=====Depth Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_DEPTH, index: 59, width: 320, height: 200, format: 8, timeStampUs: 148062180us, centerDepth: 284mm
=====Color Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_COLOR, index: 59, width: 1280, height: 720, format: 5, timeStampUs: 148145333us
=====Depth Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_DEPTH, index: 74, width: 320, height: 200, format: 8, timeStampUs: 148561816us, centerDepth: 0mm
=====Color Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_COLOR, index: 74, width: 1280, height: 720, format: 5, timeStampUs: 148645333us
[06/18 10:50:17.288758][info][17897][DeviceManager.cpp:185] Device changed! removed: 1, added: 0
[06/18 10:50:17.288883][info][17897][DeviceManager.cpp:15] Removed device(s) list: (1)
[06/18 10:50:17.288904][info][17897][DeviceManager.cpp:24]  - Name: Orbbec Gemini 2, PID: 0x0670, SN/ID: AY35C3200W2, Connection: USB2.0
[06/18 10:50:17.288921][info][17897][DeviceManager.cpp:15] Current device(s) list: (0)
Device disconnect, deviceCount: 1
stopStream vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-55
[06/18 10:50:17.289006][info][17897][Pipeline.cpp:361] Try to stop pipeline!
[06/18 10:50:17.289019][info][17897][Pipeline.cpp:333] Try to stop streams!
[06/18 10:50:17.289057][warning][17897][VideoSensor.cpp:649] Sensor(OB_SENSOR_DEPTH) already stop.
[06/18 10:50:17.289072][info][17897][Pipeline.cpp:346] Sensor stream stopped, sensorType=OB_SENSOR_DEPTH
[06/18 10:50:17.289092][warning][17897][VideoSensor.cpp:649] Sensor(OB_SENSOR_COLOR) already stop.
[06/18 10:50:17.289105][info][17897][Pipeline.cpp:346] Sensor stream stopped, sensorType=OB_SENSOR_COLOR
[06/18 10:50:17.289116][info][17897][Pipeline.cpp:352] Stop streams done!
[06/18 10:50:17.289192][error][17897][ObUvcDevice.cpp:188] setXu failed, error code=-4
[06/18 10:50:17.289211][error][17897][HostProtocol.cpp:420] Set property value failed! propertyId:42, rc:{statusCode: 2, respErrorCode: 65535, msg: send control transfer failed!}
[06/18 10:50:17.289236][warning][17897][ObException.hpp:40] Request failed, statusCode: 2, msg: send control transfer failed!
[06/18 10:50:17.289671][warning][17897][AbstractDevice.cpp:1033] Execute failure! A libobsensor_exception has occurred!
     - where:1033#resetAlignMode
     - msg:Request failed, statusCode: 2, msg: send control transfer failed!
     - type:N11libobsensor12io_exceptionE
[06/18 10:50:17.289766][warning][17897][AbstractDevice.cpp:1033] set depth align failed.
[06/18 10:50:17.289906][info][17897][Pipeline.cpp:390] Stop pipeline done!
[06/18 10:50:17.289955][info][17897][Pipeline.cpp:71] Pipeline destroyed! @0x626287EF2CB0
[06/18 10:50:17.290024][info][17897][Gemini2Device.cpp:130] Gemini2 device destroyed! PID:1648, SN:AY35C3200W2, depthMode: {name: Unbinned Dense Default, checksum(hex): 6bb28ca9a0dff5553497f9eecb59f667, optionCode: NORMAL}
[06/18 10:50:17.290074][info][17897][VideoSensor.cpp:279] VideoSensor destroyed, @OB_SENSOR_COLOR
[06/18 10:50:17.290181][info][17897][VideoSensor.cpp:279] VideoSensor destroyed, @OB_SENSOR_DEPTH
[06/18 10:50:17.292951][info][17897][ObUvcDevice.cpp:109] endpoint:129
[06/18 10:50:17.292999][info][17897][ObUvcDevice.cpp:109] endpoint:129
[06/18 10:50:17.293085][info][17897][ObUvcDevice.cpp:68] uvc_close done.
[06/18 10:50:17.293106][info][17897][ObUvcDevice.cpp:70] ~ObUvcDevice done
[06/18 10:50:17.293420][info][17897][ObUvcDevice.cpp:109] endpoint:131
[06/18 10:50:17.293476][info][17897][ObUvcDevice.cpp:109] endpoint:131
[06/18 10:50:17.293603][info][17897][ObUvcDevice.cpp:68] uvc_close done.
[06/18 10:50:17.293644][info][17897][ObUvcDevice.cpp:70] ~ObUvcDevice done
Device disconnected. vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-55
Device connect, deviceCount: 0
[06/18 10:50:23.539562][info][17977][DeviceManager.cpp:185] Device changed! removed: 0, added: 1
[06/18 10:50:23.539622][info][17977][DeviceManager.cpp:15] Current device(s) list: (1)
[06/18 10:50:23.539634][info][17977][DeviceManager.cpp:24]  - Name: Orbbec Gemini 2, PID: 0x0670, SN/ID: AY35C3200W2, Connection: USB2.0
Device disconnect, deviceCount: 0
Device connect, deviceCount: 1
[06/18 10:50:23.551191][info][17977][Gemini2Device.cpp:432] Device.OrbbecGemini2.FirmwareUpgradeFoolproof device firmware upgrade foolproof status true
[06/18 10:50:23.551951][info][17977][Gemini2Device.cpp:125] Gemini2 device created! PID:1648, SN:AY35C3200W2, depthMode: {name: Unbinned Dense Default, checksum(hex): 6bb28ca9a0dff5553497f9eecb59f667, optionCode: NORMAL}
[06/18 10:50:23.551983][info][17977][DeviceManager.cpp:154] Device created successfully! Name: Orbbec Gemini 2, PID: 0x0670, SN/ID: AY35C3200W2
[06/18 10:50:23.552031][info][17977][Pipeline.cpp:44] Pipeline created with device: {name: Orbbec Gemini 2, sn: AY35C3200W2}, @0x7CDD90010C20
Device connected. vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-56
startStream vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-56
[06/18 10:50:23.554106][info][17977][Gemini2Device.cpp:685] Color sensor has been created!
[06/18 10:50:23.559288][info][17977][Gemini2Device.cpp:637] Depth sensor has been created!
[06/18 10:50:23.560615][info][17977][Pipeline.cpp:238] Try to start streams!
[06/18 10:50:23.560646][info][17977][VideoSensor.cpp:590] start OB_SENSOR_COLOR stream with profile: {type: OB_STREAM_COLOR, format: OB_FORMAT_MJPG, width: 1280, height: 720, fps: 30}
[06/18 10:50:23.568843][info][17977][VideoSensor.cpp:590] start OB_SENSOR_DEPTH stream with profile: {type: OB_STREAM_DEPTH, format: OB_FORMAT_Y14, width: 320, height: 200, fps: 30}
[06/18 10:50:23.577637][info][17977][Pipeline.cpp:251] Start streams done!
[06/18 10:50:23.577646][info][17977][Pipeline.cpp:234] Pipeline start done!
=====Depth Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_DEPTH, index: 14, width: 320, height: 200, format: 8, timeStampUs: 66568351us, centerDepth: 284mm
=====Color Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_COLOR, index: 14, width: 1280, height: 720, format: 5, timeStampUs: 66648016us
=====Depth Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_DEPTH, index: 29, width: 320, height: 200, format: 8, timeStampUs: 67067987us, centerDepth: 284mm
=====Color Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_COLOR, index: 29, width: 1280, height: 720, format: 5, timeStampUs: 67148016us
[06/18 10:50:24.831586][info][18008][DeviceManager.cpp:185] Device changed! removed: 1, added: 0
[06/18 10:50:24.831641][info][18008][DeviceManager.cpp:15] Removed device(s) list: (1)
[06/18 10:50:24.831653][info][18008][DeviceManager.cpp:24]  - Name: Orbbec Gemini 2, PID: 0x0670, SN/ID: AY35C3200W2, Connection: USB2.0
[06/18 10:50:24.831665][info][18008][DeviceManager.cpp:15] Current device(s) list: (0)
Device disconnect, deviceCount: 1
stopStream vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-56
[06/18 10:50:24.831704][info][18008][Pipeline.cpp:361] Try to stop pipeline!
[06/18 10:50:24.831710][info][18008][Pipeline.cpp:333] Try to stop streams!
[06/18 10:50:24.831727][warning][18008][VideoSensor.cpp:649] Sensor(OB_SENSOR_DEPTH) already stop.
[06/18 10:50:24.831734][info][18008][Pipeline.cpp:346] Sensor stream stopped, sensorType=OB_SENSOR_DEPTH
[06/18 10:50:24.831744][warning][18008][VideoSensor.cpp:649] Sensor(OB_SENSOR_COLOR) already stop.
[06/18 10:50:24.831753][info][18008][Pipeline.cpp:346] Sensor stream stopped, sensorType=OB_SENSOR_COLOR
[06/18 10:50:24.831762][info][18008][Pipeline.cpp:352] Stop streams done!
[06/18 10:50:24.831800][error][18008][ObUvcDevice.cpp:188] setXu failed, error code=-4
[06/18 10:50:24.831812][error][18008][HostProtocol.cpp:420] Set property value failed! propertyId:42, rc:{statusCode: 2, respErrorCode: 65535, msg: send control transfer failed!}
[06/18 10:50:24.831824][warning][18008][ObException.hpp:40] Request failed, statusCode: 2, msg: send control transfer failed!
[06/18 10:50:24.831867][warning][18008][AbstractDevice.cpp:1033] Execute failure! A libobsensor_exception has occurred!
     - where:1033#resetAlignMode
     - msg:Request failed, statusCode: 2, msg: send control transfer failed!
     - type:N11libobsensor12io_exceptionE
[06/18 10:50:24.831880][warning][18008][AbstractDevice.cpp:1033] set depth align failed.
[06/18 10:50:24.831914][info][18008][Pipeline.cpp:390] Stop pipeline done!
[06/18 10:50:24.831927][info][18008][Pipeline.cpp:71] Pipeline destroyed! @0x7CDD90010C20
[06/18 10:50:24.831940][info][18008][Gemini2Device.cpp:130] Gemini2 device destroyed! PID:1648, SN:AY35C3200W2, depthMode: {name: Unbinned Dense Default, checksum(hex): 6bb28ca9a0dff5553497f9eecb59f667, optionCode: NORMAL}
[06/18 10:50:24.831953][info][18008][VideoSensor.cpp:279] VideoSensor destroyed, @OB_SENSOR_COLOR
[06/18 10:50:24.831979][info][18008][VideoSensor.cpp:279] VideoSensor destroyed, @OB_SENSOR_DEPTH
[06/18 10:50:24.833840][info][18008][ObUvcDevice.cpp:109] endpoint:129
[06/18 10:50:24.833848][info][18008][ObUvcDevice.cpp:109] endpoint:129
[06/18 10:50:24.833979][info][18008][ObUvcDevice.cpp:68] uvc_close done.
[06/18 10:50:24.833984][info][18008][ObUvcDevice.cpp:70] ~ObUvcDevice done
[06/18 10:50:24.834290][info][18008][ObUvcDevice.cpp:109] endpoint:131
[06/18 10:50:24.834297][info][18008][ObUvcDevice.cpp:109] endpoint:131
[06/18 10:50:24.834450][info][18008][ObUvcDevice.cpp:68] uvc_close done.
[06/18 10:50:24.834455][info][18008][ObUvcDevice.cpp:70] ~ObUvcDevice done
Device disconnected. vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-56
Device connect, deviceCount: 0
[06/18 10:50:28.823644][info][18138][DeviceManager.cpp:185] Device changed! removed: 0, added: 1
[06/18 10:50:28.823671][info][18138][DeviceManager.cpp:15] Current device(s) list: (1)
[06/18 10:50:28.823681][info][18138][DeviceManager.cpp:24]  - Name: Orbbec Gemini 2, PID: 0x0670, SN/ID: AY35C3200W2, Connection: USB2.0
Device disconnect, deviceCount: 0
Device connect, deviceCount: 1
[06/18 10:50:28.832903][info][18138][Gemini2Device.cpp:432] Device.OrbbecGemini2.FirmwareUpgradeFoolproof device firmware upgrade foolproof status true
[06/18 10:50:28.833477][info][18138][Gemini2Device.cpp:125] Gemini2 device created! PID:1648, SN:AY35C3200W2, depthMode: {name: Unbinned Dense Default, checksum(hex): 6bb28ca9a0dff5553497f9eecb59f667, optionCode: NORMAL}
[06/18 10:50:28.833500][info][18138][DeviceManager.cpp:154] Device created successfully! Name: Orbbec Gemini 2, PID: 0x0670, SN/ID: AY35C3200W2
[06/18 10:50:28.833532][info][18138][Pipeline.cpp:44] Pipeline created with device: {name: Orbbec Gemini 2, sn: AY35C3200W2}, @0x7CDD800113F0
Device connected. vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-58
startStream vid: 0x2bc5, pid: 0x670, sn: AY35C3200W2, uid: 3-7-58
[06/18 10:50:28.835251][info][18138][Gemini2Device.cpp:685] Color sensor has been created!
[06/18 10:50:28.840313][info][18138][Gemini2Device.cpp:637] Depth sensor has been created!
[06/18 10:50:28.841738][info][18138][Pipeline.cpp:238] Try to start streams!
[06/18 10:50:28.841775][info][18138][VideoSensor.cpp:590] start OB_SENSOR_COLOR stream with profile: {type: OB_STREAM_COLOR, format: OB_FORMAT_MJPG, width: 1280, height: 720, fps: 30}
[06/18 10:50:28.849513][info][18138][VideoSensor.cpp:590] start OB_SENSOR_DEPTH stream with profile: {type: OB_STREAM_DEPTH, format: OB_FORMAT_Y14, width: 320, height: 200, fps: 30}
[06/18 10:50:28.855936][info][18138][Pipeline.cpp:251] Start streams done!
[06/18 10:50:28.855941][info][18138][Pipeline.cpp:234] Pipeline start done!
=====Depth Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_DEPTH, index: 14, width: 320, height: 200, format: 8, timeStampUs: 66515002us, centerDepth: 284mm
=====Color Frame Info====== SN: AY35C3200W2, FrameType: OB_FRAME_COLOR, index: 14, width: 1280, height: 720, format: 5, timeStampUs: 66595434us

I am using an x86_64 machine running Ubuntu 22.04, connecting to the camera over USB 2.0, with SDK version 1.8.3. Any help is much appreciated and this is blocking any further development for me.

camkisailus commented 3 months ago

Just tried running HotPlugin from the latest SDK version and got the same behavior. This problem does not happen when connecting with USB 3.0, but I'm unable to do that due to hardware restrictions.

camkisailus commented 3 months ago

If it helps I am running Gemini 2 FW: 1.4.92

zhonghong322 commented 3 months ago

This is because USB 3.0 has a very low probability of being recognized as USB 2.0. Therefore, there is a feature in the firmware that enables three attempts of disconnection and reconnection when it is detected as USB 2.0. If you consistently use USB 2.0, you can disable this retry feature.

image

First, try disabling the retry feature using OrbbecViewer and see if the firmware supports it. If it does, you can also use the property interface of the Orbbec SDK to configure this setting. /**

camkisailus commented 3 months ago

@zhonghong322 Thanks for the reply! I found that actually enabling OB_PROP_DEVICE_USB2_REPEAT_IDENTIFY_BOOL to true fixes the connection problems. Here is the code I added to fix my issue:

auto devList = ctx->queryDeviceList();
    auto device = devList->getDeviceBySN(m_CameraConfig.camera_serial_number.c_str());
    if (!device)
    {
        LERROR("Failed to connect to Orbbec S/N: {}", m_CameraConfig.camera_serial_number);
        return false;
    }
    device->setBoolProperty(OBPropertyID::OB_PROP_DEVICE_USB3_REPEAT_IDENTIFY_BOOL, true);
zhonghong322 commented 3 months ago

@zhonghong322 Thanks for the reply! I found that actually enabling OB_PROP_DEVICE_USB2_REPEAT_IDENTIFY_BOOL to true fixes the connection problems. Here is the code I added to fix my issue:

auto devList = ctx->queryDeviceList();
    auto device = devList->getDeviceBySN(m_CameraConfig.camera_serial_number.c_str());
    if (!device)
    {
        LERROR("Failed to connect to Orbbec S/N: {}", m_CameraConfig.camera_serial_number);
        return false;
    }
    device->setBoolProperty(OBPropertyID::OB_PROP_DEVICE_USB3_REPEAT_IDENTIFY_BOOL, true);

yes ,you are right.

camkisailus commented 3 months ago

Thank you @zhonghong322. I have not seen any problems since making the change. This ticket is closed now :+1: