search

espressif / esp-box

The ESP-BOX is a new generation AIoT development platform released by Espressif Systems.
Apache License 2.0
710 stars 173 forks source link

Can not connect camera to Box-3 with the example: usb_camera_lcd_display (AEGHB-406) #105

Closed PaulaScharf closed 7 months ago

PaulaScharf commented 9 months ago

Hi, I am trying to run the usb_camera_lcd_display example on my new ESP32-S3-Box-3. I flashed the MCU through the USB-C on the main unit. I connected the main unit to the Dock. I tried to connect several USB-Cameras that I had laying around to the USB-Port of the Dock. But nothing happens. The display only shows me "Insert a camera, press boot for resolution". I am not sure how to debug this, because the connection to my PC through the USB-C on the main unit shuts down as soon as I connect a USB-Camera. I assume that the Pins are occupied by the camera. Do you perhaps have some insight how to resolve this? Can I connect the device in some other way to my computer to access the serial connection and possibly see error messages? Are there limitations to the kind of camera I can connect to the device? Is the UART port on the Dock or on the main unit?

I also tried to connect it the other way around, where I provide power through the USB-C on the Dock and connect the camera to the USB-C on the main unit. But the display still shows me the same message.

Any input appreciated. Thank you, Paula

espressif2022 commented 9 months ago

Hello, can you unplug the camera again, it seems that this demo will not recognize it when plugged in when powered on.

ESP-Mars commented 9 months ago

Hi, I am trying to run the usb_camera_lcd_display example on my new ESP32-S3-Box-3. I flashed the MCU through the USB-C on the main unit. I connected the main unit to the Dock. I tried to connect several USB-Cameras that I had laying around to the USB-Port of the Dock. But nothing happens. The display only shows me "Insert a camera, press boot for resolution". I am not sure how to debug this, because the connection to my PC through the USB-C on the main unit shuts down as soon as I connect a USB-Camera. I assume that the Pins are occupied by the camera.

Thanks for your feedback, which provides us invaluable improvements of our example's user guide. When burning the MCU, please through the USB-C on the main unit, and do not connect the USB Camera at the same time. After the flashing is completed, please plug the USB-C into the USB-C port on the Dock to power the USB camera. The USB-A port of the Dock is used to connect the USB camera.

Do you perhaps have some insight how to resolve this? Can I connect the device in some other way to my computer to access the serial connection and possibly see error messages?

You can access the serial connection through the U0RXD and U0TXD of the 12-pin Female Header of the Dock. We leave the UART port on the Dock. The right way to access serial is done through a standard TTL-To-USB tool.

Are there limitations to the kind of camera I can connect to the device?

Yes, the USB camera must be compatible with USB1.1 full-speed mode and support MJPEG output. Please read here to know more limitations of the USB Camera on running USB stream feature.

I also tried to connect it the other way around, where I provide power through the USB-C on the Dock and connect the camera to the USB-C on the main unit. But the display still shows me the same message.

The USB-C port on the dock doesn't support data transmission and only provide 5V power input. You should connect your camera through the USB-A port on the Dock, which supports USB Host and can provide 5V power supply to camera. Read here to know specific technical specification of the ESP32-S3-BOX-3 accessary.

PaulaScharf commented 9 months ago

Thank you for the quick responses!

When burning the MCU, please through the USB-C on the main unit, and do not connect the USB Camera at the same time. After the flashing is completed, please plug the USB-C into the USB-C port on the Dock to power the USB camera.

This was super helpful. When we do it in this order we get an image :D

685428B7-0EED-4F1D-8B45-595FD59A9CFB

It is just noise but perhaps the camera we tried is not entirely compatible. In total we tried with 5 different webcams and only one was recognized.

ESP-Mars commented 9 months ago

This was super helpful. When we follow this order we get an image :D

Yeah, this means that the camera is not compatible.

PaulaScharf commented 9 months ago

We finally found a camera that works (Logitech C930e). And we managed to get the console output through the UART Pins, by plugging them into a UM Feather S2 MCU and running this program on the feather.

However it is still confusing to us, why this camera works and the others didnt. It supports MJPEG but it is USB 2.0 which we thought was not possible, because only USB 1.1 is allowed? This is the camera config that is outputted to the console:

*** Device descriptor ***
bcdUSB 2.00
bDeviceClass 0xef
bDeviceSubClass 0x2
bDeviceProtocol 0x1
bMaxPacketSize0 64
idVendor 0x46d
idProduct 0x843
bNumConfigurations 1
I (1668) USB_STREAM: ENUM Stage CHECK_FULL_DEV_DESC, Succeed
I (1675) USB_STREAM: ENUM Stage GET_SHORT_CONFIG_DESC, Succeed
I (1682) USB_STREAM: ENUM Stage CHECK_SHORT_CONFIG_DESC, Succeed
I (1689) USB_STREAM: ENUM Stage GET_FULL_CONFIG_DESC, Succeed
*** Configuration descriptor ***
wTotalLength 1110
bNumInterfaces 4
bConfigurationValue 1
*** Interface Association Descriptor: Video ***
        *** Interface descriptor ***
        bInterfaceNumber 0
        bAlternateSetting 0
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x1
                *** Endpoint descriptor ***
                bEndpointAddress 0x86   EP 6 IN
                bmAttributes 0x3        INT
                wMaxPacketSize 64
                bInterval 16
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 0
        bNumEndpoints 0
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
        *** Class-specific VS Interface Descriptor ***
        bNumFormats 2
        *** VS Format MJPEG Descriptor ***
        bFormatIndex 0x2
        bNumFrameDescriptors 9
        bDefaultFrameIndex 1
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x1
        wWidth 640
        wHeigh 480
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x2
        wWidth 160
        wHeigh 120
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x3
        wWidth 176
        wHeigh 144
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x4
        wWidth 320
        wHeigh 180
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x5
        wWidth 320
        wHeigh 240
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x6
        wWidth 352
        wHeigh 288
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x7
        wWidth 424
        wHeigh 240
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x8
        wWidth 480
        wHeigh 270
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x9
        wWidth 640
        wHeigh 360
        FrameInterval[0] 333333
        FrameInterval[1] 416666
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 1333333
        FrameInterval[6] 2000000
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 1
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 192
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 2
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 384
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 3
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 512
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 4
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 640
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 5
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 800
                bInterval 1
*** Interface Association Descriptor: Audio ***
        *** Interface descriptor ***
        bInterfaceNumber 2
        bAlternateSetting 0
        bNumEndpoints 0
        bInterfaceClass 0x1 (Audio)
        bInterfaceSubClass 0x1
        *** Audio control header descriptor ***
        bcdADC 0x100
        wTotalLength 38
        bInCollection 1
                Interface number[0] = 3
        *** Audio control input terminal descriptor ***
        bTerminalID 1
        wTerminalType 0x201
        bAssocTerminal 0
        bNrChannels 1
        wChannelConfig 0x0003
        *** Audio control output terminal descriptor ***
        bTerminalID 3
        wTerminalType 0x101
        bAssocTerminal 0
        *** Audio control feature unit descriptor ***
        bUnitID 5
        bSourceID 1
        bControlSize 1
        bmaControls[ch0] 0x3
        *** Interface descriptor ***
        bInterfaceNumber 3
        bAlternateSetting 0
        bNumEndpoints 0
        bInterfaceClass 0x1 (Audio)
        bInterfaceSubClass 0x2
        *** Interface descriptor ***
        bInterfaceNumber 3
        bAlternateSetting 1
        bNumEndpoints 1
        bInterfaceClass 0x1 (Audio)
        bInterfaceSubClass 0x2
        *** Audio stream general descriptor ***
        bTerminalLink 3
        bDelay 1
        wFormatTag 1
        *** Audio control header descriptor ***
        bFormatType 1
        bNrChannels 2
        bSubframeSize 2
        bBitResolution 16
        bSamFreqType 1
        tSamFreq[0] 16000
                *** Endpoint descriptor ***
                bEndpointAddress 0x83   EP 3 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 68
                bInterval 4

And this is the config of a camera that doesnt work:

*** Device descriptor ***
bcdUSB 2.00
bDeviceClass 0xef
bDeviceSubClass 0x2
bDeviceProtocol 0x1
bMaxPacketSize0 64
idVendor 0x46d
idProduct 0x825
bNumConfigurations 1
I (3474) USB_STREAM: ENUM Stage CHECK_FULL_DEV_DESC, Succeed
I (3482) USB_STREAM: ENUM Stage GET_SHORT_CONFIG_DESC, Succeed
I (3488) USB_STREAM: ENUM Stage CHECK_SHORT_CONFIG_DESC, Succeed
I (3495) USB_STREAM: ENUM Stage GET_FULL_CONFIG_DESC, Succeed
*** Configuration descriptor ***
wTotalLength 1117
bNumInterfaces 4
bConfigurationValue 1
*** Interface Association Descriptor: Video ***
        *** Interface descriptor ***
        bInterfaceNumber 0
        bAlternateSetting 0
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x1
                *** Endpoint descriptor ***
                bEndpointAddress 0x87   EP 7 IN
                bmAttributes 0x3        INT
                wMaxPacketSize 16
                bInterval 16
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 0
        bNumEndpoints 0
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
        *** Class-specific VS Interface Descriptor ***
        bNumFormats 3
        *** VS Format MJPEG Descriptor ***
        bFormatIndex 0x2
        bNumFrameDescriptors 12
        bDefaultFrameIndex 1
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x1
        wWidth 640
        wHeigh 480
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x2
        wWidth 160
        wHeigh 120
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x3
        wWidth 176
        wHeigh 144
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x4
        wWidth 320
        wHeigh 176
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x5
        wWidth 320
        wHeigh 240
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x6
        wWidth 352
        wHeigh 288
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x7
        wWidth 432
        wHeigh 240
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x8
        wWidth 544
        wHeigh 288
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0x9
        wWidth 640
        wHeigh 360
        FrameInterval[0] 333333
        FrameInterval[1] 400000
        FrameInterval[2] 500000
        FrameInterval[3] 666666
        FrameInterval[4] 1000000
        FrameInterval[5] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0xa
        wWidth 752
        wHeigh 416
        FrameInterval[0] 666666
        FrameInterval[1] 1000000
        FrameInterval[2] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0xb
        wWidth 800
        wHeigh 448
        FrameInterval[0] 666666
        FrameInterval[1] 1000000
        FrameInterval[2] 2000000
        *** VS MJPEG Frame Descriptor ***
        bFrameIndex 0xc
        wWidth 800
        wHeigh 600
        FrameInterval[0] 666666
        FrameInterval[1] 1000000
        FrameInterval[2] 2000000
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 1
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 192
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 2
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 384
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 3
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 512
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 4
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 640
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 5
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 800
                bInterval 1
        *** Interface descriptor ***
        bInterfaceNumber 1
        bAlternateSetting 6
        bNumEndpoints 1
        bInterfaceClass 0xe (Video)
        bInterfaceSubClass 0x2
                *** Endpoint descriptor ***
                bEndpointAddress 0x81   EP 1 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 944
                bInterval 1
*** Interface Association Descriptor: Audio ***
        *** Interface descriptor ***
        bInterfaceNumber 2
        bAlternateSetting 0
        bNumEndpoints 0
        bInterfaceClass 0x1 (Audio)
        bInterfaceSubClass 0x1
        *** Audio control header descriptor ***
        bcdADC 0x100
        wTotalLength 38
        bInCollection 1
                Interface number[0] = 3
        *** Audio control input terminal descriptor ***
        bTerminalID 1
        wTerminalType 0x201
        bAssocTerminal 0
        bNrChannels 1
        wChannelConfig 0x0000
        *** Audio control output terminal descriptor ***
        bTerminalID 3
        wTerminalType 0x101
        bAssocTerminal 1
        *** Audio control feature unit descriptor ***
        bUnitID 5
        bSourceID 1
        bControlSize 1
        bmaControls[ch0] 0x3
        bmaControls[ch1] 0x0
        *** Interface descriptor ***
        bInterfaceNumber 3
        bAlternateSetting 0
        bNumEndpoints 0
        bInterfaceClass 0x1 (Audio)
        bInterfaceSubClass 0x2
        *** Interface descriptor ***
        bInterfaceNumber 3
        bAlternateSetting 1
        bNumEndpoints 1
        bInterfaceClass 0x1 (Audio)
        bInterfaceSubClass 0x2
        *** Audio stream general descriptor ***
        bTerminalLink 3
        bDelay 1
        wFormatTag 1
        *** Audio control header descriptor ***
        bFormatType 1
        bNrChannels 1
        bSubframeSize 2
        bBitResolution 16
        bSamFreqType 1
        tSamFreq[0] 16000
                *** Endpoint descriptor ***
                bEndpointAddress 0x86   EP 6 IN
                bmAttributes 0x5        ISOC
                wMaxPacketSize 68
                bInterval 1

The configs seem extremely similar, but somehow, with the second camera we get a core panic, which the logs suggest is because an interface setting with a wMaxPacketSize of 944 is selected:

I (5747) uvc_camera_lcd_demo: Device connected
I (5747) USB_STREAM: Creating UVC pipe: ifc=1-3, ep=0x81, mps=512
I (5748) USB_STREAM: Probe Format(2) MJPEG, Frame(1) 640*480, interval(666666)
I (5756) USB_STREAM: Probe payload size = 512
bFormatIndex: 2
bFrameIndex: 1
dwFrameInterval: 666666
dwMaxPayloadTransferSize: 944
bInterfaceNumber: 0
I (5866) USB_STREAM: dwMaxPayloadTransferSize set = 512, probed = 944
I (5866) USB_STREAM: Sample processing task start
I (5866) USB_STREAM: Set Device Interface = 1, Alt = 3
I (5877) USB_STREAM: Set Device Interface Done
I (5878) USB_STREAM: UVC Streaming...
I (5883) USB_STREAM: USB Device Connected
I (5888) main_task: Returned from app_main()
I (8956) uvc_camera_lcd_demo: uvc callback! frame_format = 7, seq = 1, width = 640, height = 480, length = 17488, ptr = 0
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.

With the working camera the selected wMaxPacketSize seems to be 384. Is it possible to select which interface setting is used in the code? Because the camera that caused the core panic, seems to have other interface settings available. S

leeebo commented 9 months ago

@PaulaScharf I don't think the problem was caused by wMaxPacketSize because we got success in set Device Interface = 1, Alt = 3, which means the actual MPS is 512. And the first frame I (8956) uvc_camera_lcd_demo: uvc callback! frame_format = 7, seq = 1, width = 640, height = 480, length = 17488, ptr = 0 is got successfully!

Please try with a specified resolution like .frame_width = 320 and .frame_height = 240 through the configs to find what will happen

Maybe there is a bug in the example's camera_frame_cb, you can comment out the code usb_camera_lcd_display/main/main.c#L135C4-L144, to check if the camera itself works.

PaulaScharf commented 8 months ago

Thank you again! When I change the width and height in the config, I now also get a picture with a webcam, that was previously not working.

ESP-Mars commented 7 months ago

Temporarily closed due to the extended period of time with no response. Should you have additional questions or concerns, don't hesitate to reopen the issue.