search

IntelRealSense / RealSenseID

Intel® RealSense™ ID SDK
https://intelrealsense.com/facial-authentication/
Apache License 2.0
98 stars 56 forks source link

Failed receiving data packet on Raspberry Pi 4 with Ubuntu 22.04 #150

Open ZeosBios opened 2 years ago

ZeosBios commented 2 years ago

I have Raspberry Pi 4 (aarch64 Cortex-A72) with OS Ubuntu 22.04 LTS 64-bit installed. I downloaded RealSenseID Version 0.25.0 and then build with cmake .. -DRSID_PREVIEW=1 -DRSID_SECURE=1. When I run ./rsid-cli /dev/ttyACM0 and execute commands I got errors:

pi@pi:/usr/local/rsid/build/bin$ ./rsid-cli /dev/ttyACM0
[2022-08-15 18:23:41.122] [debug] [LinuxSerial] Opening serial port /dev/ttyACM0 baudrate 115200
Connected to device
[2022-08-15 18:23:41.190] [debug] [SecureSession] Start session
[2022-08-15 18:23:41.204] [debug] [PacketSender] Sending packet 'c'
[2022-08-15 18:23:41.205] [debug] [PacketSender] Waiting packet..
[2022-08-15 18:23:41.215] [error] [PacketSender] Got invalid crc. Expected: 561. Actual: 20100
[2022-08-15 18:23:41.215] [error] [SecureSession] Failed to recv device key response
[2022-08-15 18:23:41.215] [error] [FaceAuthenticatorImpl] Session start failed with status 108
Status: CrcError

[2022-08-15 18:23:41.215] [debug] [SecureSession] Close session
Please select an option:

  'e' to enroll.
  'a' to authenticate.
  'd' to delete all users.
  'p' to pair with the device (enables secure communication).
  'i' to unpair with the device (disables secure communication).
  'c' to capture images from device.
  's' to set authentication settings.
  'g' to query authentication settings.
  'u' to query ids of users.
  'n' to query number of users.
  'b' to save device's database before standby.
  'v' to view additional information.
  'x' to ping the device.
  'q' to quit.

server mode options:
  'E' to enroll with faceprints.
  'A' to authenticate with faceprints.
  'U' to list enrolled users
  'D' to delete all users.

> i
[2022-08-15 18:23:46.864] [debug] [LinuxSerial] Opening serial port /dev/ttyACM0 baudrate 115200
Connected to device
[2022-08-15 18:23:46.868] [debug] [PacketSender] Sending packet 'a'
[2022-08-15 18:23:46.869] [debug] [PacketSender] Waiting packet..
[2022-08-15 18:23:46.879] [error] [PacketSender] Got invalid crc. Expected: 561. Actual: 20100
[2022-08-15 18:23:46.879] [error] [SecureSession] Failed to recv device ecdsa public key
Failed to unpair with device
[2022-08-15 18:23:46.879] [debug] [SecureSession] Close session
Please select an option:

  'e' to enroll.
  'a' to authenticate.
  'd' to delete all users.
  'p' to pair with the device (enables secure communication).
  'i' to unpair with the device (disables secure communication).
  'c' to capture images from device.
  's' to set authentication settings.
  'g' to query authentication settings.
  'u' to query ids of users.
  'n' to query number of users.
  'b' to save device's database before standby.
  'v' to view additional information.
  'x' to ping the device.
  'q' to quit.

server mode options:
  'E' to enroll with faceprints.
  'A' to authenticate with faceprints.
  'U' to list enrolled users
  'D' to delete all users.

> p
[2022-08-15 18:23:48.456] [debug] [LinuxSerial] Opening serial port /dev/ttyACM0 baudrate 115200
Connected to device
[2022-08-15 18:23:48.458] [info] [FaceAuthenticatorImpl] Pairing start
[2022-08-15 18:23:48.458] [debug] [PacketSender] Sending packet 'a'
[2022-08-15 18:23:48.460] [debug] [PacketSender] Waiting packet..
[2022-08-15 18:23:48.469] [error] [PacketSender] Got invalid crc. Expected: 561. Actual: 20100
[2022-08-15 18:23:48.469] [error] [FaceAuthenticatorImpl] Failed to recv device ecdsa public key
Failed pairing with device
[2022-08-15 18:23:48.469] [debug] [SecureSession] Close session

While trying to make capture:

Please select an option:

  'e' to enroll.
  'a' to authenticate.
  'd' to delete all users.
  'p' to pair with the device (enables secure communication).
  'i' to unpair with the device (disables secure communication).
  'c' to capture images from device.
  's' to set authentication settings.
  'g' to query authentication settings.
  'u' to query ids of users.
  'n' to query number of users.
  'b' to save device's database before standby.
  'v' to view additional information.
  'x' to ping the device.
  'q' to quit.

server mode options:
  'E' to enroll with faceprints.
  'A' to authenticate with faceprints.
  'U' to list enrolled users
  'D' to delete all users.

> c
[2022-08-15 18:23:50.712] [debug] [Utilities] capture devices 1
starting preview for 3 seconds [2022-08-15 18:23:50.721] [debug] [Preview] Preview started!
[2022-08-15 18:23:50.787] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x01 0x80
[2022-08-15 18:23:50.886] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x03 0x80
[2022-08-15 18:23:50.952] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x03 0x81
[2022-08-15 18:23:51.018] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x02 0x80
[2022-08-15 18:23:51.085] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x04 0x80
[2022-08-15 18:23:51.152] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x05 0x80
[2022-08-15 18:23:51.218] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x02 0x80
[2022-08-15 18:23:51.285] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x02 0x80
[2022-08-15 18:23:51.351] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x02 0x80
[2022-08-15 18:23:51.418] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x03 0x80
[2022-08-15 18:23:51.485] [debug] [StreamConverter] jpeg error: Not a JPEG file: starts with 0x02 0x80

When I try update firmware:

pi@pi:/usr/local/rsid/build/bin$ ./rsid-fw-update --file F450_4.3.0.8200_SKU1_SIGNED.bin  --port /dev/ttyACM0 --force-version
Using manual device selection...

[2022-08-15 18:31:49.678] [debug] [LinuxSerial] Opening serial port /dev/ttyACM0 baudrate 115200
[2022-08-15 18:31:50.006] [info] [FwUpdater] Extract modules from "F450_4.3.0.8200_SKU1_SIGNED.bin"
[2022-08-15 18:31:50.006] [debug] [FwUpdater] [    OPFW] 2.76 MB,  6 blocks
[2022-08-15 18:31:50.053] [debug] [FwUpdater] [   NNLED] 2.64 MB,  6 blocks
[2022-08-15 18:31:50.095] [debug] [FwUpdater] [   NNLAS] 2.28 MB,  5 blocks
[2022-08-15 18:31:50.127] [debug] [FwUpdater] [    DNET] 2.95 MB,  6 blocks
[2022-08-15 18:31:50.174] [debug] [FwUpdater] [    YOLO] 1.04 MB,  3 blocks
[2022-08-15 18:31:50.188] [debug] [FwUpdater] [   RECOG] 4.32 MB,  9 blocks
[2022-08-15 18:31:50.257] [debug] [FwUpdater] [  AS2DLR] 2.30 MB,  5 blocks
[2022-08-15 18:31:50.289] [info] [FwUpdater] Extracted 7 modules
[2022-08-15 18:31:50.289] [debug] [FwUpdater] 
Device does not support the encryption applied on the firmware. Replace firmware binary.

pi@pi:/usr/local/rsid/build/bin$ ./rsid-fw-update --file F450_4.3.0.8200_SKU2_SIGNED.bin --port /dev/ttyACM0
Using manual device selection...

[2022-08-15 18:32:50.622] [debug] [LinuxSerial] Opening serial port /dev/ttyACM0 baudrate 115200
[2022-08-15 18:32:51.086] [info] [FwUpdater] Extract modules from "F450_4.3.0.8200_SKU2_SIGNED.bin"
[2022-08-15 18:32:51.086] [debug] [FwUpdater] [    OPFW] 2.76 MB,  6 blocks
[2022-08-15 18:32:51.134] [debug] [FwUpdater] [   NNLED] 2.64 MB,  6 blocks
[2022-08-15 18:32:51.175] [debug] [FwUpdater] [   NNLAS] 2.28 MB,  5 blocks
[2022-08-15 18:32:51.207] [debug] [FwUpdater] [    DNET] 2.95 MB,  6 blocks
[2022-08-15 18:32:51.254] [debug] [FwUpdater] [    YOLO] 1.04 MB,  3 blocks
[2022-08-15 18:32:51.268] [debug] [FwUpdater] [   RECOG] 4.32 MB,  9 blocks
[2022-08-15 18:32:51.337] [debug] [FwUpdater] [  AS2DLR] 2.30 MB,  5 blocks
[2022-08-15 18:32:51.369] [info] [FwUpdater] Extracted 7 modules
[2022-08-15 18:32:51.369] [debug] [FwUpdater] 

Summary:
 * Serial number: 12056228170799A27FM10464-100
 * Serial port: /dev/ttyACM0
 * Incompatible with current device firmware
 * Compatible with new device firmware
 * Firmware update path:
     * OPFW: 2.5.0.74 -> 4.3.0.8200
     * RECOG: 28.2.21.0 -> 1.6.21.0

[2022-08-15 18:32:51.371] [debug] [LinuxSerial] Opening serial port /dev/ttyACM0 baudrate 115200
[2022-08-15 18:32:51.824] [info] [FwUpdater] Extract modules from "F450_4.3.0.8200_SKU2_SIGNED.bin"
[2022-08-15 18:32:51.824] [debug] [FwUpdater] [    OPFW] 2.76 MB,  6 blocks
[2022-08-15 18:32:51.869] [debug] [FwUpdater] [   NNLED] 2.64 MB,  6 blocks
[2022-08-15 18:32:51.906] [debug] [FwUpdater] [   NNLAS] 2.28 MB,  5 blocks
[2022-08-15 18:32:51.938] [debug] [FwUpdater] [    DNET] 2.95 MB,  6 blocks
[2022-08-15 18:32:51.980] [debug] [FwUpdater] [    YOLO] 1.04 MB,  3 blocks
[2022-08-15 18:32:51.995] [debug] [FwUpdater] [   RECOG] 4.32 MB,  9 blocks
[2022-08-15 18:32:52.059] [debug] [FwUpdater] [  AS2DLR] 2.30 MB,  5 blocks
[2022-08-15 18:32:52.091] [info] [FwUpdater] Extracted 7 modules
[2022-08-15 18:32:52.091] [debug] [FwUpdater] 
Firmware cannot be updated directly to the chosen version.
Flash firmware version 3.1.#.# first.

Info about my firmware and device:

> v
[2022-08-15 18:35:14.875] [debug] [LinuxSerial] Opening serial port /dev/ttyACM0 baudrate 115200

Additional information:
 * S/N: 12056228170799A27FM10464-100
 * Firmware: OPFW:2.5.0.74|NNLED:2.22.21.0|NNLAS:2.12.21.0|DNET:4.5.20.0|RECOG:28.2.21.0|YOLO:25.4.21.0|AS2DLR:23.2.21.0
 * Host: 0.25.0

UPD:

After debug src files I detected error in LinuxCapture.cc. I added logs in buffer V4lNode::Read() function:

    if (ioctl(_fd, VIDIOC_DQBUF, &buf) == FAILED_V4L){ // dequeue frame from buffer.
        std::cout << "V4lNode failed  dequeue frame from buffer: " << strerror(errno) << std::endl;
        return res; //returns zero-length buffer
    }

Output result: V4lNode failed dequeue frame from buffer: Resource temporarily unavailable

But when I try open device:

V4lNode::V4lNode(int camera_number,v4l2_format format):_type(format.type)
....
        // open device
        std::string dev_md = VIDEO_DEV + std::to_string(camera_number);
        _fd = open(dev_md.c_str(), O_RDWR | O_NONBLOCK, 0);
        std::cout << "V4lNode open files discription: " << strerror(errno) << std::endl;
        ThrowIfFailed("fd_md", _fd);

Output result: V4lNode open files discription: Success

When I run V4L tests:

pi@pi:~$ v4l2-compliance 
v4l2-compliance 1.22.1, 64 bits, 64-bit time_t

Compliance test for uvcvideo device /dev/video0:

Driver Info:
    Driver name      : uvcvideo
    Card type        : Intel F450
    Bus info         : usb-0000:01:00.0-1.1
    Driver version   : 5.15.39
    Capabilities     : 0x84a00001
        Video Capture
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04200001
        Video Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : uvcvideo
    Model            : Intel F450
    Serial           : 00.00.01
    Bus info         : usb-0000:01:00.0-1.1
    Media version    : 5.15.39
    Hardware revision: 0x00000100 (256)
    Driver version   : 5.15.39
Interface Info:
    ID               : 0x03000002
    Type             : V4L Video
Entity Info:
    ID               : 0x00000001 (1)
    Name             : Intel F450
    Function         : V4L2 I/O
    Flags            : default
    Pad 0x01000007   : 0: Sink
      Link 0x02000010: from remote pad 0x100000a of entity 'Extension 3' (Video Pixel Formatter): Data, Enabled, Immutable

Required ioctls:
    test MC information (see 'Media Driver Info' above): OK
    test VIDIOC_QUERYCAP: OK
    test invalid ioctls: OK

Allow for multiple opens:
    test second /dev/video0 open: OK
    test VIDIOC_QUERYCAP: OK
    test VIDIOC_G/S_PRIORITY: OK
    test for unlimited opens: OK

Debug ioctls:
    test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
    test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
    test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
    test VIDIOC_ENUMAUDIO: OK (Not Supported)
    test VIDIOC_G/S/ENUMINPUT: OK
    test VIDIOC_G/S_AUDIO: OK (Not Supported)
    Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
    test VIDIOC_G/S_MODULATOR: OK (Not Supported)
    test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
    test VIDIOC_ENUMAUDOUT: OK (Not Supported)
    test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
    test VIDIOC_G/S_AUDOUT: OK (Not Supported)
    Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
    test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
    test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
    test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
    test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
        fail: v4l2-test-controls.cpp(246): missing control class for class 00980000
    test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: FAIL
    test VIDIOC_QUERYCTRL: OK
    test VIDIOC_G/S_CTRL: OK
        warn: v4l2-test-controls.cpp(757): g_ext_ctrls(0) invalid error_idx 0
    test VIDIOC_G/S/TRY_EXT_CTRLS: OK
    test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
    test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
    Standard Controls: 1 Private Controls: 0

Format ioctls (Input 0):
    test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
    test VIDIOC_G/S_PARM: OK
    test VIDIOC_G_FBUF: OK (Not Supported)
    test VIDIOC_G_FMT: OK
    test VIDIOC_TRY_FMT: OK
        warn: v4l2-test-formats.cpp(1036): Could not set fmt2
    test VIDIOC_S_FMT: OK
    test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
    test Cropping: OK (Not Supported)
    test Composing: OK (Not Supported)
    test Scaling: OK (Not Supported)

Codec ioctls (Input 0):
    test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
    test VIDIOC_G_ENC_INDEX: OK (Not Supported)
    test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK (Not Supported)

Total for uvcvideo device /dev/video0: 46, Succeeded: 45, Failed: 1, Warnings: 2

Outputs formats:

pi@pi:~$ v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'YUYV' (YUYV 4:2:2)

Output dmesg errors:

[    0.823576] zswap: compressor zstd not available, using default lzo
[    0.829609] zswap: zpool z3fold not available, using default zbud
[  102.238315] uvcvideo 1-1.1:1.1: Failed to resubmit video URB (-1).
[  102.238626] uvcvideo 1-1.1:1.1: Failed to resubmit video URB (-1).
[ 5822.239686] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5822.249854] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5822.259913] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5822.269949] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5822.280057] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5822.290139] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5822.391565] usb 1-1.1: Failed to query (SET_CUR) UVC control 4 on unit 1: -32 (exp. 4).
[ 5822.401637] usb 1-1.1: Failed to query (SET_CUR) UVC control 4 on unit 1: -32 (exp. 4).
[ 5822.411679] usb 1-1.1: Failed to query (SET_CUR) UVC control 4 on unit 1: -32 (exp. 4).
[ 5822.421697] usb 1-1.1: Failed to query (SET_CUR) UVC control 4 on unit 1: -32 (exp. 4).
[ 5822.431779] usb 1-1.1: Failed to query (SET_CUR) UVC control 4 on unit 1: -32 (exp. 4).
[ 5822.441885] usb 1-1.1: Failed to query (SET_CUR) UVC control 4 on unit 1: -32 (exp. 4).
[ 5822.532650] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5822.563375] usb 1-1.1: Failed to query (SET_CUR) UVC control 4 on unit 1: -32 (exp. 4).
[ 5822.622056] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5823.168241] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).
[ 5823.270367] usb 1-1.1: Failed to query (SET_CUR) UVC control 10 on unit 2: -32 (exp. 2).

Do you have an ideas?