search

groupgets / purethermal1-uvc-capture

USB Video Class capture examples for PureThermal 1 / PureThermal 2 FLIR Lepton Dev Kit
125 stars 81 forks source link

Lepton 3.5 radiometric, purethermal2 smartIO board on Ubuntu 18.04.2LTS #15

Closed okumurah1103 closed 4 years ago

okumurah1103 commented 5 years ago

Hi colleagues,

I already use purethermal2 smart IO board with Lepton3.5 radiometric and uvc-radiometry.py works very well on Mac OSX High Sierra. However, when I use the same script (the same version of Anaconda 3 environment,and latest libusb and latest libuvc) on UBUNTU 18.04.2LTS, the script outputs the following error and don't work. The LEPTON device seems to be recognized normally, but image frames can be captured. Please tell me the way to overcome this issue?

sudo python uvc-radiometry.py

device opened! Version gpp: 3.3.26 dsp: 3.3.26 FLIR part #: 500-0771-01 FLIR serial #: '\xea\xf3(\x00\x00\x00\x00\x00' format: UYVY frame 160x120 @ 9fps format: Y16 frame 160x120 @ 9fps frame 160x122 @ 9fps format: Y8
frame 160x120 @ 9fps format: RGBP frame 160x120 @ 9fps format: }�6� frame 160x120 @ 9fps Estimated / selected altsetting bandwith : 18 / 962. uvc_start_streaming failed: -1 <===HERE

kekiefer commented 5 years ago

Are you using the GroupGets fork of libuvc?

okumurah1103 commented 5 years ago

Thank you for your suggestion. I use latest repository version on GitHub. Best regard.

Prof. H.Okumura Saga University Japan

2019年6月22日(土) 0:05 Kurt Kiefer notifications@github.com:

Are you using the GroupGets fork of libuvc https://github.com/groupgets/libuvc?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/groupgets/purethermal1-uvc-capture/issues/15?email_source=notifications&email_token=AJHDD4OOJJUJFEVKQE5CQHTP3TU3LA5CNFSM4H2O6EYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYIXDFY#issuecomment-504459671, or mute the thread https://github.com/notifications/unsubscribe-auth/AJHDD4NDSLNZB7I2F7YKFT3P3TU3LANCNFSM4H2O6EYA .

okumurah1103 commented 5 years ago

Dear sir.

In addition, I try libuvc under GetThermal directory on GroupGets, but the result is exactly the same to latest version on GitHub. Thanks.

2019年6月22日(土) 0:05 Kurt Kiefer notifications@github.com:

Are you using the GroupGets fork of libuvc https://github.com/groupgets/libuvc?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/groupgets/purethermal1-uvc-capture/issues/15?email_source=notifications&email_token=AJHDD4OOJJUJFEVKQE5CQHTP3TU3LA5CNFSM4H2O6EYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYIXDFY#issuecomment-504459671, or mute the thread https://github.com/notifications/unsubscribe-auth/AJHDD4NDSLNZB7I2F7YKFT3P3TU3LANCNFSM4H2O6EYA .

kekiefer commented 5 years ago

There are only two places in libuvc that return -1 (UVC_ERROR_IO), and they're both in device.c during initialization (failures calling libusb_get_config_descriptor or libusb_get_device_list), so neither of them are called by uvc_start_streaming.

That leads us to that your -1 is coming from libusb (which is LIBUSB_ERROR_IO). It's hard to say why this might fail, as there are lots of possibilities. I suggest that you start with a web search and see what it turns up: https://www.google.com/search?hl=en&q=LIBUSB_ERROR_IO

okumurah1103 commented 5 years ago

Dear sir.

Thanks your effective advice. I will check libuvc source and try it again.

HIRO

2019年6月22日(土) 3:11 Kurt Kiefer notifications@github.com:

There are only two places in libuvc that return -1 (UVC_ERROR_IO), and they're both in device.c during initialization (failures calling libusb_get_config_descriptor or libusb_get_device_list), so neither of them are called by uvc_start_streaming.

That leads us to that your -1 is coming from libusb (which is LIBUSB_ERROR_IO). It's hard to say why this might fail, as there are lots of possibilities. I suggest that you start with a web search and see what it turns up: https://www.google.com/search?hl=en&q=LIBUSB_ERROR_IO

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/groupgets/purethermal1-uvc-capture/issues/15?email_source=notifications&email_token=AJHDD4J2FS2PSHRWBRFDCPDP3UKTNA5CNFSM4H2O6EYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYJGAUQ#issuecomment-504520786, or mute the thread https://github.com/notifications/unsubscribe-auth/AJHDD4L33SHALQEA4PIL4V3P3UKTNANCNFSM4H2O6EYA .

okumurah1103 commented 5 years ago

Dear sir.

Do you have recommend libusb version for PT2-UVC-capture? I have installed v.1.0.22 on GitHub. Thank you for your precise assistance.

HIRO

2019年6月22日(土) 3:11 Kurt Kiefer notifications@github.com:

There are only two places in libuvc that return -1 (UVC_ERROR_IO), and they're both in device.c during initialization (failures calling libusb_get_config_descriptor or libusb_get_device_list), so neither of them are called by uvc_start_streaming.

That leads us to that your -1 is coming from libusb (which is LIBUSB_ERROR_IO). It's hard to say why this might fail, as there are lots of possibilities. I suggest that you start with a web search and see what it turns up: https://www.google.com/search?hl=en&q=LIBUSB_ERROR_IO

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/groupgets/purethermal1-uvc-capture/issues/15?email_source=notifications&email_token=AJHDD4J2FS2PSHRWBRFDCPDP3UKTNA5CNFSM4H2O6EYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYJGAUQ#issuecomment-504520786, or mute the thread https://github.com/notifications/unsubscribe-auth/AJHDD4L33SHALQEA4PIL4V3P3UKTNANCNFSM4H2O6EYA .

okumurah1103 commented 5 years ago

Dear Sir:

Other note PC + Ubuntu16.04 32bit + libusb1.0.22 + libuvc on /ithub = uvc-radiometry.py works well. Apple MacBook Air + libusb1.0.22 + libuvc on /ithub = uvc-radiometry.py works well too.

However both of

BTO handheld mini PC + Ubuntu16.04 64bit + libusb1.0.22 + libuvc on /ithub BTO handheld mini PC + Ubuntu18.04 64bit + libusb1.0.22 + libuvc on /ithub

do not work with the same error message uvc_start_streaming failed: -1

ummmmm.... is it a hardware depend problem?? (;_;)

kekiefer commented 5 years ago

It does sound like it could be a hardware problem. The first two things I would take a look at:

Is the PC's USB port supplying enough power. do you see a continuous 5V to the PT board (you will have to probe the board with an oscilliscope or a high quality DVM)?

Does the PC have an internal USB hub that is being shared with other devices?

okumurah1103 commented 5 years ago

Thanks

I use self powered USB hub for LEPTON camera module. I will talk about it with developer for this BTO PC. Thanks for your kindly assistance.

HIROSHI OKUMURA,PH.D

2019年6月29日(土) 0:07 Kurt Kiefer notifications@github.com:

It does sound like it could be a hardware problem. The first two things I would take a look at:

Is the PC's USB port supplying enough power. do you see a continuous 5V to the PT board (you will have to probe the board with an oscilliscope or a high quality DVM)?

Does the PC have an internal USB hub that is being shared with other devices?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/groupgets/purethermal1-uvc-capture/issues/15?email_source=notifications&email_token=AJHDD4N6IYIMZEMPP4TBSTTP4YSJPA5CNFSM4H2O6EYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2KWUA#issuecomment-506768208, or mute the thread https://github.com/notifications/unsubscribe-auth/AJHDD4O7PTW3VGKUAH7QHRLP4YSJPANCNFSM4H2O6EYA .

abramowitzK commented 5 years ago

I'm getting the same issue on Ubuntu 18.0.4.2. Tried unplugging all the other devices on usb. Will try to verify power.

The light on the board goes from flashing slowly to staying lit for about a second and then going back to flashing slowing. I've tested the board on another computer and works fine.

Sometimes it does start to flash quickly for a second too but then quickly goes back to slow flashing

abramowitzK commented 5 years ago

I did some digging. My problem appears to be that the Purethermal is appearing as a USB1.1 device instead of a 2.0 device and is being limited to 12Mbps. This is causing VIDIOC_STREAMON: No space left on device Also in dmesg: 

[ 2357.613887] usb 1-1.2: new full-speed USB device number 10 using ci_hdrc
[ 2357.724910] usb 1-1.2: New USB device found, idVendor=1e4e, idProduct=0100
[ 2357.724924] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2357.724931] usb 1-1.2: Product: PureThermal (fw:v1.2.2)
[ 2357.724937] usb 1-1.2: Manufacturer: GroupGets
[ 2357.724943] usb 1-1.2: SerialNumber: 8013001e-5102-3038-3835-393400000000
[ 2357.726367] uvcvideo: Found UVC 1.00 device PureThermal (fw:v1.2.2) (1e4e:0100)
[ 2357.727836] uvcvideo 1-1.2:1.0: Entity type for entity Extension 3 was not initialized!
[ 2357.727858] uvcvideo 1-1.2:1.0: Entity type for entity Processing 2 was not initialized!
[ 2357.727870] uvcvideo 1-1.2:1.0: Entity type for entity Extension 4 was not initialized!
[ 2357.727882] uvcvideo 1-1.2:1.0: Entity type for entity Extension 5 was not initialized!
[ 2357.727891] uvcvideo 1-1.2:1.0: Entity type for entity Extension 6 was not initialized!
[ 2357.727900] uvcvideo 1-1.2:1.0: Entity type for entity Extension 7 was not initialized!
[ 2357.727909] uvcvideo 1-1.2:1.0: Entity type for entity Extension 21 was not initialized!
[ 2357.727918] uvcvideo 1-1.2:1.0: Entity type for entity Extension 254 was not initialized!
[ 2357.727927] uvcvideo 1-1.2:1.0: Entity type for entity Camera 1 was not initialized!
[ 2357.728592] input: PureThermal (fw:v1.2.2) as /devices/soc0/soc/2100000.aips-bus/2184200.usb/ci_hdrc.1/usb1/1-1/1-1.2/1-1.2:1.0/input/input17
[ 2359.113844] uvcvideo: Failed to submit URB 0 (-28).

This only happens on the ARM board I'm trying to run on and not on my desktop

lsusb -t

/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ci_hdrc/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 2: Dev 10, If 0, Class=Video, Driver=uvcvideo, 12M
        |__ Port 2: Dev 10, If 1, Class=Video, Driver=uvcvideo, 12M
kekiefer commented 5 years ago

It is correct that the PT boards operate at USB Full Speed (12 Mbps), not High Speed (480 Mbps). So the port they're connected to must also operate at this speed. As such, the hub you're connecting them to must support per-port transaction translation if you're going to connect multiple devices to it, otherwise you won't have enough bandwidth to use the devices simultaneously.

abramowitzK commented 5 years ago

This happens even with 1 device though

That lsusb -t I listed above is from having only 1 board plugged in

kekiefer commented 5 years ago

OK, I see.

I searched Google for your error: https://www.google.com/search?hl=en&q=uvcvideo%3A%20Failed%20to%20submit%20URB%200%2028

In these results, I see lots of people having issues specific to their platform's USB drivers and USB webcams in general. It doesn't appear this problem is specific to the PureThermal board. What platform is this on?

Can the OP comment on where we stand with the original issue? It's not clear to me if this is the same problem manifested differently because of using different capture drivers (libuvc vs uvcvideo/v4l2) or not.

abramowitzK commented 5 years ago

Thanks for looking into it. I've combed through those results. I guess I'll have to do some digging into the drivers. This is on a Gateworks Ventana GW5224 SBC with a PCI usb hub. Ubuntu 18.0.4.2. I'll write back if I find anything.

Thanks!

kekiefer commented 5 years ago

See if you can rule out the mini PCIe hub and connect directly to a port on the root hub instead. Another angle might try using a different PCIe USB card, one with its own controller so it has its own root port and a different driver as well. They're a little harder to find, but here's one https://www.amazon.com/StarTech-com-SuperSpeed-Express-Adapter-Bracket/dp/B00DDL88OS

abramowitzK commented 5 years ago

Thanks for the advice! I get the same issue if I'm connected to the root hub. I will look into trying a pci card with it's own root port. This feels like an issue with libusb but I'm not entirely sure.

I've tried a different webcam (Logitech C920) that supports compressed video and it doesn't seem to have this problem. I'm thinking that the compression has something to do with it working?

The actual error I'm getting is:

VIDIOC_STREAMON error 28, No space left on device

which is coming from: 

uvcvideo: Failed to submit URB 0 (-28).

Which is coming from libusb:

usb 1-1.2: usbfs: usb_submit_urb returned -28

Still digging...

kekiefer commented 5 years ago

Unless I haven't had enough coffee, it doesn't look to me like you using the libusb/libuvc stack for capture. uvcvideo is the kernel usb v4l2 driver, which would be used by gstreamer or opencv's videocapture module, for example. It would be interesting to know if you used e.g. the python radiometry code if you still have the same problem.

You should use lsusb to look at the usb descriptors for you Logitech cam. One possible difference could be that the Logitech cam uses bulk endpoints for data rather than isochronous. The problem will be specific to the iso endpoints, because with these the system is responsible to set aside enough bandwidth to run the stream, and that's what your error messages are all about.

abramowitzK commented 5 years ago

Ah yes sorry those errors above were from me trying to use v4l2 in my debugging. However, when running the python radiometry code I get similar errors:

usb 1-1: usbfs: usb_submit_urb returned -28

Is what I get when I try to run the examples in python using the groupgets libuvc

Thanks for the info about isochronous vs bulk endpoints. I will look into this as well. Is there a good way to debug/see how much bandwidth the purethermal is requesting? Or a way to limit it?

kekiefer commented 5 years ago

The bandwidth is based on how much data must be transferred to capture a particular stream, which is a combination of the frame format and frame rate. This reported by the PureThermal's firmware in the UVC descriptors. If you capture a color space like YUV420 (12 bits-per-pixel), it will be less than RGB (24 bits-per-pixel). You could conceivably modify the firmware to report any value you wanted for the bandwidth for all these streams.

abramowitzK commented 5 years ago

I see lsusb gives:

        dwMinBitRate                  2764800
        dwMaxBitRate                  2764800
        dwMaxVideoFrameBufferSize       38400

For the format I'm using, is this the value that it uses for bandwidth? It is seems low enough to be okay...

This is correct from my calculations too (120 160 16bpp * 9fps)

kekiefer commented 5 years ago

Yes... and therein lies the problem. Why would your kernel drivers be rejecting this? Is there a different OS distribution you can easily try?

I have access to an iMX6 system running freescale's vendor kernel 4.9.123, but I don't have a PT board with me. When I get a chance I'll try it out.

abramowitzK commented 5 years ago

Thanks for looking into it, I really appreciate it. I've tried linux 4.10 and just flashed a new image on here with 4.20.7. Both were ubuntu 18.04.2 though. I'll see if I can grab a different flavor of linux and get this running on there.

abramowitzK commented 5 years ago

Not sure if this is useful:

[ 1043.461218] uvcvideo: Failed to submit URB 0 (-28).
[ 1045.686840] uvcvideo: Device requested 962 B/frame bandwidth.
[ 1045.686855] uvcvideo: Selecting alternate setting 1 (962 B/frame bandwidth).
[ 1045.693005] uvcvideo: Allocated 5 URB buffers of 32x962 bytes each.
[ 1045.693801] uvcvideo: Failed to submit URB 0 (-28).
[ 1209.840209] uvcvideo: uvc_v4l2_open
[ 1209.936385] uvcvideo: Resuming interface 0
[ 1209.936419] uvcvideo: Resuming interface 1
[ 1209.938366] uvcvideo: uvc_v4l2_mmap
[ 1209.938708] uvcvideo: uvc_v4l2_mmap
[ 1209.938882] uvcvideo: uvc_v4l2_mmap
[ 1209.939052] uvcvideo: uvc_v4l2_mmap
[ 1209.940357] uvcvideo: Device requested 962 B/frame bandwidth.
[ 1209.940387] uvcvideo: Selecting alternate setting 1 (962 B/frame bandwidth).
[ 1209.955275] uvcvideo: Allocated 5 URB buffers of 32x962 bytes each.
[ 1209.956228] uvcvideo: Failed to submit URB 0 (-28).
[ 1209.964020] uvcvideo: uvc_v4l2_release
[ 1211.992835] uvcvideo: Suspending interface 1
[ 1211.992938] uvcvideo: Suspending interface 0

When trying to run using v4l2 and C with bandwidth tracing enabled

kekiefer commented 5 years ago

Can you repeat that on a system that works?

kekiefer commented 5 years ago

By the way, in case you haven't figured this already -28 is -ENOSPC. This is from the linux kernel usb error-codes doc:

-ENOSPC     This request would overcommit the usb bandwidth reserved
        for periodic transfers (interrupt, isochronous).
abramowitzK commented 5 years ago

Thanks yea I have gathered that by now. Here's from my desktop:

[532091.363461] uvcvideo: uvc_v4l2_open
[532091.499143] uvcvideo: Resuming interface 0
[532091.499145] uvcvideo: Resuming interface 1
[532091.499512] uvcvideo: uvc_v4l2_mmap
[532091.499521] uvcvideo: uvc_v4l2_mmap
[532091.499527] uvcvideo: uvc_v4l2_mmap
[532091.499533] uvcvideo: uvc_v4l2_mmap
[532091.499845] uvcvideo: Device requested 962 B/frame bandwidth.
[532091.499847] uvcvideo: Selecting alternate setting 1 (962 B/frame bandwidth).
[532091.500868] uvcvideo: Allocated 5 URB buffers of 32x962 bytes each.
[532091.500923] uvcvideo: uvc_v4l2_poll
[532091.884918] uvcvideo: Frame complete (EOF found).
[532091.885006] uvcvideo: uvc_v4l2_poll
[532091.896305] uvcvideo: uvc_v4l2_release
[532094.010561] uvcvideo: Suspending interface 1
[532094.010563] uvcvideo: Suspending interface 0
kekiefer commented 5 years ago

More digging through the kernel. Doesn't look like a silver bullet (considering you don't have ANY other devices enabled -- unless that hub counts somehow?)... but maybe the following option would help? In fact, if you have it enabled, it looks like maybe disabling it might even help.

config USB_EHCI_TT_NEWSCHED
    bool "Improved Transaction Translator scheduling"
    depends on USB_EHCI_HCD
    default y
    ---help---
      This changes the periodic scheduling code to fill more of the low
      and full speed bandwidth available from the Transaction Translator
      (TT) in USB 2.0 hubs.  Without this, only one transfer will be
      issued in each microframe, significantly reducing the number of
      periodic low/fullspeed transfers possible.

      If you have multiple periodic low/fullspeed devices connected to a
      highspeed USB hub which is connected to a highspeed USB Host
      Controller, and some of those devices will not work correctly
      (possibly due to "ENOSPC" or "-28" errors), say Y.  Conversely, if
      you have only one such device and it doesn't work, you could try
      saying N.

      If unsure, say Y.
abramowitzK commented 5 years ago

Interesting...I can try removing the PCI hub I have on there and just trying on the root hub without that enabled. Also will check this option! I tried using the uvcvideo quirk FIX_BANDWIDTH too and that seemed to not do anything

EDIT:

No dice on removing the usb hub.

abramowitzK commented 5 years ago

I'm trying ubuntu 14.04 now with a freescale kernel version 3.14 with the pt grab from this repo

[ 1296.563786] uvcvideo: uvc_v4l2_open
[ 1296.636847] uvcvideo: Resuming interface 0
[ 1296.636869] uvcvideo: Resuming interface 1
[ 1296.639270] uvcvideo: uvc_v4l2_ioctl(VIDIOC_QUERYCAP)
[ 1296.639441] uvcvideo: uvc_v4l2_ioctl(VIDIOC_QUERYCAP)
[ 1296.639474] uvcvideo: uvc_v4l2_ioctl(VIDIOC_G_FMT)
[ 1296.639498] uvcvideo: uvc_v4l2_ioctl(VIDIOC_G_PARM)
[ 1296.639590] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FMT)
[ 1296.639639] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639670] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639689] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FMT)
[ 1296.639714] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639741] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639758] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639776] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FMT)
[ 1296.639793] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639811] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639828] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FMT)
[ 1296.639844] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639861] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FRAMESIZES)
[ 1296.639878] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FMT)
[ 1296.639896] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUM_FMT)
[ 1296.639914] uvcvideo: uvc_v4l2_ioctl(VIDIOC_QUERYCAP)
[ 1296.639968] uvcvideo: uvc_v4l2_ioctl(VIDIOC_G_INPUT)
[ 1296.639998] uvcvideo: uvc_v4l2_ioctl(VIDIOC_ENUMINPUT)
[ 1296.641911] uvcvideo: uvc_v4l2_ioctl(VIDIOC_QUERYCTRL)
[ 1296.642145] uvcvideo: uvc_v4l2_ioctl(VIDIOC_TRY_FMT)
[ 1296.642183] uvcvideo: Trying format 0x20363159 (Y16 ): 160x120.
[ 1296.642204] uvcvideo: Using default frame interval 111111.1 us (9.0 fps).
[ 1296.646856] uvcvideo: uvc_v4l2_ioctl(VIDIOC_S_FMT)
[ 1296.646895] uvcvideo: Trying format 0x20363159 (Y16 ): 160x120.
[ 1296.646914] uvcvideo: Using default frame interval 111111.1 us (9.0 fps).
[ 1296.651825] uvcvideo: uvc_v4l2_ioctl(VIDIOC_G_PARM)
[ 1296.652090] uvcvideo: uvc_v4l2_ioctl(VIDIOC_REQBUFS)
[ 1296.652385] uvcvideo: uvc_v4l2_ioctl(VIDIOC_QUERYBUF)
[ 1296.652439] uvcvideo: uvc_v4l2_mmap
[ 1296.652602] uvcvideo: uvc_v4l2_ioctl(VIDIOC_QUERYBUF)
[ 1296.652636] uvcvideo: uvc_v4l2_mmap
[ 1296.652795] uvcvideo: uvc_v4l2_ioctl(VIDIOC_QBUF)
[ 1296.652828] uvcvideo: uvc_v4l2_ioctl(VIDIOC_QBUF)
[ 1296.652852] uvcvideo: uvc_v4l2_ioctl(VIDIOC_STREAMON)
[ 1296.653823] uvcvideo: Device requested 962 B/frame bandwidth.
[ 1296.653841] uvcvideo: Selecting alternate setting 1 (962 B/frame bandwidth).
[ 1296.656781] uvcvideo: Allocated 5 URB buffers of 32x962 bytes each.
[ 1296.656937] uvcvideo: Failed to submit URB 0 (-28).
[ 1296.663928] uvcvideo: uvc_v4l2_release
[ 1298.995482] uvcvideo: Suspending interface 1
[ 1298.995501] uvcvideo: Suspending interface 0
agjunyent commented 5 years ago

A similar issue is happening to me, using a Raspberry Pi 4 with the newest kernel of Raspbian Buster:

https://github.com/groupgets/purethermal1-firmware/issues/24

I've still haven't figured what is causing the error.

Gateworks commented 4 years ago

I've posted a question about this -ENOSPC issue to the linux-media and linux-usb mailing lists: https://www.mail-archive.com/linux-media@vger.kernel.org/msg150265.html

kekiefer commented 4 years ago

Thanks for posting that, it's probably the right place. Your follow-up response is consistent with our findings and those from raspberrypi/linux:

The -ENOSPC is getting returned from drivers/usb/host/ehci-sched.c:iso_stream_schedule()

Here are a couple more cross-references for this issue: https://github.com/raspberrypi/linux/issues/3136 https://github.com/groupgets/purethermal1-firmware/issues/24

Gateworks commented 4 years ago

Just following up here with an explanation from the linux-usb mailing list:

The asks for an 8-byte interrupt endpoint together with a 962-byte isochronous endpoint.

USB 2.0 host controllers are supported by the Linux ehci-hcd driver (the same code manages any EHCI host controller) which doesn't do a good job of handling high-bandwidth periodic requests for full-speed devices, particularly when isochronous and interrupt endpoints are both present.

While the amount of data being requested (8-byte interrupt and 962-byte iso) is fine for full-speed USB (up to 1023 byte frames), handling full-speed devices attached to a high-speed controller isn't easy, and ehci-hcd isn't able to make use of all the possible bandwidth. In fact, you'd be better off attaching the camera to a full-speed USB 1.1 controller, if one were available for your system.

USB 3.0 (xHCI) controllers, on the other hand, handle all the scheduling issues in hardware so the driver doesn't have to deal with them. Evidently the ones you tried don't have any trouble in this situation.

The same problem will occur whenever this device is attached to an EHCI controller on a Linux-based system, unless somebody goes to the trouble of improving the driver. (It's tremendously complicated -- the spec puts almost all the burden on the software rather than the hardware/firmware -- and probably not worth the effort of trying to do it correctly)

The easy solution (if your not a USB expert and want to take on improving the ehci driver bandwidth scheduling) would be to move to a board with a USB 1.0 host controller or a USB 3.0 host controller or use an add-in card that provides this.

Its too bad the camera doesn't offer any fallback endpoints using smaller USB frames as many devices do as that would probably make it work as well.

kekiefer commented 4 years ago

Thanks for the update.

Well it's not quite true that most XHCI controllers work, considering that the new raspberry pi has one, and it doesn't work there. But that just makes the problem harder to fix, because then it's the host controller firmware that's broken!

We can add some additional fallback endpoints that will satisfy the need for 16-bit radiometric data (let's face it, this is what most people are interested in) but I don't think the RGB data formats will be possible.

Gateworks commented 4 years ago

True... XHCI does reservation in hardware so you could end up with buggy hardware/firmware on the host controller side.

After realizing this was an open-source design with open-source firmware (cool!) I built the firmware with the hack to lower the USB frame size at https://github.com/groupgets/purethermal1-firmware/issues/24 and re-flashed the PT I have here. The endpoint now reports an endpoint with MxPS=482 as expected but when I try to capture a single frame via 'v4l2-ctl --device $DEV --stream-mmap --stream-to=x.raw --stream-count=1' instead of failing with the ENOSPC it hangs indefinitely. Any ideas what's going on here?

Maybe I'm building the wrong firmware or something because after updating the camera via DFU v4l2 shows: ~# v4l2-ctl --device $DEV --all Driver Info (not using libv4l2): Driver name : uvcvideo Card type : PureThermal (fw:v1.2.2): PureTh Bus info : usb-ci_hdrc.1-1.4 Driver version: 5.3.0 Capabilities : 0x84A00001 Video Capture Metadata Capture Streaming Extended Pix Format Device Capabilities Device Caps : 0x04200001 Video Capture Streaming Extended Pix Format Priority: 2 Video input : 0 (Camera 1: ok) Format Video Capture: Width/Height : 80/60 Pixel Format : 'UYVY' Field : None Bytes per Line : 160 Size Image : 9600 Colorspace : sRGB Transfer Function : Default (maps to sRGB) YCbCr/HSV Encoding: Default (maps to ITU-R 601) Quantization : Default (maps to Limited Range) Flags : Crop Capability Video Capture: Bounds : Left 0, Top 0, Width 80, Height 60 Default : Left 0, Top 0, Width 80, Height 60 Pixel Aspect: 1/1 Selection: crop_default, Left 0, Top 0, Width 80, Height 60 Selection: crop_bounds, Left 0, Top 0, Width 80, Height 60 Streaming Parameters Video Capture: Capabilities : timeperframe Frames per second: 9.000 (9/1) Read buffers : 0 brightness 0x00980900 (int) : min=0 max=255 step=1 default=128 value=128 contrast 0x00980901 (int) : min=0 max=255 step=1 default=128 value=128

The previous driver version on the PT I have here was 4.14.4 and reported as 160x120 YUV

kekiefer commented 4 years ago

The frame size for the descriptors is determined when the firmware first loads (and is fixed until it restarts), so I would check that you have your sensor fully seated. I suspect that should fix the 80x60 size reporting here.

I wouldn't expect any issues, but just be aware that my test with the reduced packet size leveraged the libuvc backend and the Y16 data type. YUV is a 16-bit data type, just like the raw IR data (in the Y16 UVC type), so the stream should integrate ok with the 482 byte packet.

Gateworks commented 4 years ago

Bingo! Thank you... the sensor definitely came a bit unseated from the board when my fat fingers thrashed about trying to get the BOOT and RST buttons pressed in the right order (not easy!). So I can now capture/stream fine using the 482 byte USB frames.

Will you be able to update the firmware with some fallbacks as a better solution?

kekiefer commented 4 years ago

I think at this point, it seems to be the only way forward since there are reports of buggy EHCI drivers and buggy XHCI firmware everywhere now. If it's straightforward (I think it is) I'll hammer it out as soon as I can.

kekiefer commented 4 years ago

I've made a change to the development branch to enable a lower bandwidth altsetting groupgets/purethermal1-firmware@15f64742a6a1998c7ff63ba4d67f5524e332a371. It's working properly for me with V4L2. Libuvc needed an update for me, so I'm going to work on upstreaming the few patches that are left. Let me know how it works out for you.

Gateworks commented 4 years ago

@abramowitzK can you test the latest firmware? We no longer have the camera here.

abramowitzK commented 3 years ago

Thanks for testing this all out and getting that patch in. @kekiefer You mentioned needing to update libuvc. What branch should I pull down for that? Do I still have to run the groupgets/libuvc?

kekiefer commented 3 years ago

You should use https://github.com/groupgets/purethermal1-uvc-capture/tree/master+libuvc-upstream and the upsteam https://github.com/libuvc/libuvc. I have https://github.com/groupgets/purethermal1-uvc-capture/pull/34 open to make this switch the default, and should probably merge that already.

abramowitzK commented 3 years ago

Thanks!

abramowitzK commented 3 years ago

So interestingly With FW version 1.3 and upstream libuvc it works on my PC. On my ARM board, I get to the uvc_start_streaming call and it actually starts streaming kinda. The light starts flashing quickly like it's capturing images at least but it seems I never exit uvc_start_streaming and my callback never gets called.

This might be me doing something dumb though so still investigating.

Also getting a usbfs: usb_submit_urb returned -28 in dmesg

abramowitzK commented 3 years ago

even unplugging the camera doesn't exit uvc_start_streaming. odd...