search

groupgets / GetThermal

Cross-platform USB thermal camera viewer
MIT License
228 stars 90 forks source link

No video on the GUI #29

Closed alduxvm closed 4 years ago

alduxvm commented 4 years ago

Hi all,

I'm using a lepton 2.0, the pt2 and an ubuntu laptop.

I successfully compiled GetThermal after a lot of issues and missing dependencies with qt. When I open the device using VLC it does work and I can see the thermal image from /dev/video1, but when I execute:

sudo ./GetThermal

I see the next screen:

error-getthermal

The output in the terminal is:

QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (1e4e:0100/8011002a-5119-3138-3931-383900000000) ---
Status: idle
VideoControl:
    bcdUVC: 0x0100
VideoStreaming(1):
    bEndpointAddress: 129
    Formats:
    UncompressedFormat(1)
          bits per pixel: 16
          GUID: 5559565900001000800000aa00389b71 (UYVY)
          default frame: 1
          aspect ratio: 0x0
          interlace flags: 00
          copy protect: 00
            FrameDescriptor(1)
              capabilities: 02
              size: 80x60
              bit rate: 691200-691200
              max frame size: 9600
              default interval: 1/9
              interval[0]: 1/9
    UncompressedFormat(2)
          bits per pixel: 16
          GUID: 5931362000001000800000aa00389b71 (Y16 )
          default frame: 1
          aspect ratio: 0x0
          interlace flags: 00
          copy protect: 00
            FrameDescriptor(1)
              capabilities: 02
              size: 80x60
              bit rate: 691200-691200
              max frame size: 9600
              default interval: 1/9
              interval[0]: 1/9
            FrameDescriptor(2)
              capabilities: 02
              size: 80x63
              bit rate: 725760-725760
              max frame size: 10080
              default interval: 1/9
              interval[0]: 1/9
    UncompressedFormat(3)
          bits per pixel: 8
          GUID: 5938202000001000800000aa00389b71 (Y8  )
          default frame: 1
          aspect ratio: 0x0
          interlace flags: 00
          copy protect: 00
            FrameDescriptor(1)
              capabilities: 02
              size: 80x60
              bit rate: 345600-345600
              max frame size: 4800
              default interval: 1/9
              interval[0]: 1/9
    UncompressedFormat(4)
          bits per pixel: 16
          GUID: 5247425000001000800000aa00389b71 (RGBP)
          default frame: 1
          aspect ratio: 0x0
          interlace flags: 00
          copy protect: 00
            FrameDescriptor(1)
              capabilities: 02
              size: 80x60
              bit rate: 691200-691200
              max frame size: 9600
              default interval: 1/9
              interval[0]: 1/9
    UncompressedFormat(5)
          bits per pixel: 24
          GUID: 7deb36e44f52ce119f530020af0ba770 (}�6�OR��S)
          default frame: 1
          aspect ratio: 0x0
          interlace flags: 00
          copy protect: 00
            FrameDescriptor(1)
              capabilities: 02
              size: 80x60
              bit rate: 1036800-1036800
              max frame size: 14400
              default interval: 1/9
              interval[0]: 1/9
END DEVICE CONFIGURATION
Initializing lepton SDK with UVC backend...
Using GroupGets PureThermal (fw:v1.3.0) with firmware 8011002a-5119-3138-3931-383900000000
OK
Found extension unit ID 3, controls: 000fffff, GUID: 70 74 31 2d 6c 65 70 2d 61 67 63 2d 30 30 30 30
Found extension unit ID 4, controls: 7fffffbf, GUID: 70 74 31 2d 6c 65 70 2d 6f 65 6d 2d 30 30 30 30
Found extension unit ID 5, controls: 3ffcffc81ffffff, GUID: 70 74 31 2d 6c 65 70 2d 72 61 64 2d 30 30 30 30
Found extension unit ID 6, controls: 007fffff, GUID: 70 74 31 2d 6c 65 70 2d 73 79 73 2d 30 30 30 30
Found extension unit ID 7, controls: 00003fff, GUID: 70 74 31 2d 6c 65 70 2d 76 69 64 2d 30 30 30 30
Found extension unit ID 21, controls: 07ff9ff9, GUID: 70 74 31 2d 6c 65 70 2d 72 61 64 32 30 30 30 30
Found extension unit ID 254, controls: 0000003f, GUID: 70 74 31 2d 6c 65 70 2d 63 75 73 74 30 30 30 30
bmHint: 0000
bFormatIndex: 5
bFrameIndex: 1
dwFrameInterval: 1111111
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 0
dwMaxVideoFrameSize: 14400
dwMaxPayloadTransferSize: 962
bInterfaceNumber: 1
UVC_GetAttribute failed: -9Estimated / selected altsetting bandwith : 13 / 642. 
Streaming...
QQmlExpression: Expression qrc:/ViewerForm.ui.qml:68:22 depends on non-NOTIFYable properties:
    LeptonVariation::supportsRadiometry
QQmlExpression: Expression qrc:/ViewerForm.ui.qml:53:30 depends on non-NOTIFYable properties:
    LeptonVariation::supportsRadiometry
UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9Surface set. Supported formats: 18 19 22 23 20 21 3 1 10 8 5
Surface supports format 3 width 80 height 60QQmlExpression: Expression qrc:/lepton/InfoControls.qml:51:23 depends on non-NOTIFYable properties:
    LeptonVariation::ptFirmwareVersion
QQmlExpression: Expression qrc:/lepton/InfoControls.qml:46:23 depends on non-NOTIFYable properties:
    LeptonVariation::oemDspSoftwareVersion
QQmlExpression: Expression qrc:/lepton/InfoControls.qml:41:23 depends on non-NOTIFYable properties:
    LeptonVariation::oemGppSoftwareVersion
QQmlExpression: Expression qrc:/lepton/InfoControls.qml:36:23 depends on non-NOTIFYable properties:
    LeptonVariation::sysFlirSerialNumber
QQmlExpression: Expression qrc:/lepton/InfoControls.qml:31:23 depends on non-NOTIFYable properties:
    LeptonVariation::oemFlirPartNumber
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcHeqNormalizationFactor value: 1
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcHeqEmptyCount value: 2
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcHeqMidPoint value: 128
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcHeqBinExtension value: 0
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcHeqClipLimitLow value: 512
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcHeqClipLimitHigh value: 4800
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcHeqMaxGain value: 1
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcHeqDampingFactor value: 64
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcLinearDampeningFactor value: 1
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcLinearMidPoint value: 128
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcLinearMaxGain value: 1
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcLinearHistogramClipPercent value: 0
qml: model: LeptonVariation(0x55ad490cadd0, "LeptonVariation") property: agcLinearHistogramTailSize value: 0
QQmlExpression: Expression qrc:/lepton/AgcControls.qml:11:14 depends on non-NOTIFYable properties:
    LeptonVariation::supportsRadiometry
QQmlExpression: Expression qrc:/lepton/VidControls.qml:69:22 depends on non-NOTIFYable properties:
    LeptonVariation::supportsRadiometry
QQmlExpression: Expression qrc:/lepton/VidControls.qml:62:22 depends on non-NOTIFYable properties:
    LeptonVariation::supportsRadiometry
QQmlExpression: Expression qrc:/lepton/VidControls.qml:47:22 depends on non-NOTIFYable properties:
    LeptonVariation::supportsHwPseudoColor
    LeptonVariation::supportsRadiometry
QQmlExpression: Expression qrc:/lepton/VidControls.qml:26:22 depends on non-NOTIFYable properties:
    LeptonVariation::supportsHwPseudoColor
    LeptonVariation::supportsRadiometry
qrc:/lepton/VidControls.qml:149: TypeError: Cannot read property 'data' of undefined
qrc:/lepton/VidControls.qml:143: TypeError: Cannot read property 'data' of undefined
UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_SetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9UVC_GetAttribute failed: -9Done streaming.
Device closed
UVC exited

I'm not sure if its a problem of a missing qt dependency or a problem with the compilation or with my camera...

kekiefer commented 4 years ago

This application uses libuvc, rather than v4l2, to get data from the camera. That said, the output looks mostly OK, other than all this UVC_GetAttribute failed: -9. I wonder if this is something with the 2.0 core?

You can try the python uvc-capture.py example from here https://github.com/groupgets/purethermal1-uvc-capture -- this also uses libuvc, but does not use QT, so it will cut out a third of the problem at least.

alduxvm commented 4 years ago

thanks for the fast reply!

Testing python opencv-capture.py from the purethermal1-uvc-capture works normally:

working

kekiefer commented 4 years ago

Can you check that GetThermal is using the same version of libuvc, like if you have a different version installed that it is picking up instead? LD_DEBUG=libs might help with this

alduxvm commented 4 years ago

Where can I check the LD_DEBUG? in the GetThermal.pro or the Makefile thats inside build?

alduxvm commented 4 years ago

Strangely I tried to do it in a mac, and got the same issue... the white screen...

kekiefer commented 4 years ago

I dug up a 2.0 camera core and replicated this issue on the master branch. I tried the master+upstream-libuvc branch, and it worked.

There were a few libuvc fixes related to the multiple altsettings that we implemented for the 1.3.0 firmware, and GetThermal uses a higher bandwidth altsetting for RGB than the uvc capture examples, so I'm guessing this is the problem.

You'll want to build and install libuvc from our fork for MacOS, but for linux you can use the upstream libuvc.

Please let me know how the build goes for you, it might be time to merge these changes into master.

alduxvm commented 4 years ago

I found a strange behaviour when I was doing some tests...

Using the breakout board v1.4 and a raspberry, using the LeptonModule I get this camera behaviour when I turn on a lighter close to the camera:

https://i.imgur.com/UGfiLcV.mp4

IMAGE 2020-04-07 14:15:49

And when I tried the breakout board 2.0 via usb to the raspberry pi and using vlc to record, trying the same experiment, the camera behaves different!!:

https://i.imgur.com/j3SgtwE.mp4

IMAGE 2020-04-07 14:16:02

Is this common? or my lepton core is damaged? what do you think?

kekiefer commented 4 years ago

This is working as it should do, and might clear up a comment I made above about GetThermal trying to grab the higher bandwidth RGB channel rather than Y16.

Your first video is grabbing Y16 (the raw thermal data, though it isn't radiometric on your 2.0 core) and applying a linear AGC across the whole image in software, and colorizing with the "ironblack" palette also in software.

Your second video is FLIR's hardware AGC that does a type of adaptive histogram equalization to improve contrast throughout the image, and also applying a different color palette in hardware. The camera returns data in 24-bit RGB format to support, and it is color downsampled for YUV420 or YUV422 that VLC will deal with. This image will look slightly better in GetThermal, because it will grab the original RGB stream, but this is what requires the high bandwidth altsetting.

https://lepton.flir.com/application-notes/basic-agc-for-radiometric-lepton-images

alduxvm commented 4 years ago

Thanks for the great explanation!

This is the output of GetThermal using the PT2 on the RPI: https://i.imgur.com/iw3qWuT.mp4 There are some lines on the right side of the screen... but the output does look slightly better as you mention... When trying to deactivate the AGC in the GUI, the frame froze...

Also, I tried again in my ubuntu 18.04 computer, I compiled and installed the libusb version from the github directly and then proceed to compile GetThermal, but got this error:

g++ -c -pipe -O2 -g -std=gnu++11 -flto -fno-fat-lto-objects -Wall -W -Wall -D_REENTRANT -fPIC -D__STDC_LIMIT_MACROS -DGIT_VERSION="\"v0.1.4\"" -D_TTY_NOWARN_ -DQT_NO_DEBUG -DQT_NO_DEBUG -DQT_QUICK_LIB -DQT_MULTIMEDIA_LIB -DQT_GUI_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -I../../GetThermal -I. -I../lepton_sdk/Inc -I/home/aldux/GetThermal/libuvc/build/include -I/home/aldux/GetThermal/libuvc/include -I../inc -isystem /usr/include/libusb-1.0 -isystem /usr/include/x86_64-linux-gnu/qt5 -isystem /usr/include/x86_64-linux-gnu/qt5/QtQuick -isystem /usr/include/x86_64-linux-gnu/qt5/QtMultimedia -isystem /usr/include/x86_64-linux-gnu/qt5/QtGui -isystem /usr/include/x86_64-linux-gnu/qt5/QtQml -isystem /usr/include/x86_64-linux-gnu/qt5/QtNetwork -isystem /usr/include/x86_64-linux-gnu/qt5/QtCore -Imoc -isystem /usr/include/libdrm -I/usr/lib/x86_64-linux-gnu/qt5/mkspecs/linux-g++ -o obj/main.o ../src/main.cpp
g++ -c -pipe -O2 -g -std=gnu++11 -flto -fno-fat-lto-objects -Wall -W -Wall -D_REENTRANT -fPIC -D__STDC_LIMIT_MACROS -DGIT_VERSION="\"v0.1.4\"" -D_TTY_NOWARN_ -DQT_NO_DEBUG -DQT_NO_DEBUG -DQT_QUICK_LIB -DQT_MULTIMEDIA_LIB -DQT_GUI_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -I../../GetThermal -I. -I../lepton_sdk/Inc -I/home/aldux/GetThermal/libuvc/build/include -I/home/aldux/GetThermal/libuvc/include -I../inc -isystem /usr/include/libusb-1.0 -isystem /usr/include/x86_64-linux-gnu/qt5 -isystem /usr/include/x86_64-linux-gnu/qt5/QtQuick -isystem /usr/include/x86_64-linux-gnu/qt5/QtMultimedia -isystem /usr/include/x86_64-linux-gnu/qt5/QtGui -isystem /usr/include/x86_64-linux-gnu/qt5/QtQml -isystem /usr/include/x86_64-linux-gnu/qt5/QtNetwork -isystem /usr/include/x86_64-linux-gnu/qt5/QtCore -Imoc -isystem /usr/include/libdrm -I/usr/lib/x86_64-linux-gnu/qt5/mkspecs/linux-g++ -o obj/uvcvideoproducer.o ../src/uvcvideoproducer.cpp
g++ -c -pipe -O2 -g -std=gnu++11 -flto -fno-fat-lto-objects -Wall -W -Wall -D_REENTRANT -fPIC -D__STDC_LIMIT_MACROS -DGIT_VERSION="\"v0.1.4\"" -D_TTY_NOWARN_ -DQT_NO_DEBUG -DQT_NO_DEBUG -DQT_QUICK_LIB -DQT_MULTIMEDIA_LIB -DQT_GUI_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -I../../GetThermal -I. -I../lepton_sdk/Inc -I/home/aldux/GetThermal/libuvc/build/include -I/home/aldux/GetThermal/libuvc/include -I../inc -isystem /usr/include/libusb-1.0 -isystem /usr/include/x86_64-linux-gnu/qt5 -isystem /usr/include/x86_64-linux-gnu/qt5/QtQuick -isystem /usr/include/x86_64-linux-gnu/qt5/QtMultimedia -isystem /usr/include/x86_64-linux-gnu/qt5/QtGui -isystem /usr/include/x86_64-linux-gnu/qt5/QtQml -isystem /usr/include/x86_64-linux-gnu/qt5/QtNetwork -isystem /usr/include/x86_64-linux-gnu/qt5/QtCore -Imoc -isystem /usr/include/libdrm -I/usr/lib/x86_64-linux-gnu/qt5/mkspecs/linux-g++ -o obj/uvcacquisition.o ../src/uvcacquisition.cpp
../src/uvcacquisition.cpp: In member function ‘void UvcAcquisition::setVideoFormat(const QVideoSurfaceFormat&)’:
../src/uvcacquisition.cpp:153:21: error: ‘UVC_FRAME_FORMAT_I420’ was not declared in this scope
         uvcFormat = UVC_FRAME_FORMAT_I420;
                     ^~~~~~~~~~~~~~~~~~~~~
../src/uvcacquisition.cpp:153:21: note: suggested alternative: ‘UVC_FRAME_FORMAT_NV12’
         uvcFormat = UVC_FRAME_FORMAT_I420;
                     ^~~~~~~~~~~~~~~~~~~~~
                     UVC_FRAME_FORMAT_NV12
../src/uvcacquisition.cpp:159:21: error: ‘UVC_FRAME_FORMAT_Y16’ was not declared in this scope
         uvcFormat = UVC_FRAME_FORMAT_Y16;
                     ^~~~~~~~~~~~~~~~~~~~
../src/uvcacquisition.cpp:159:21: note: suggested alternative: ‘UVC_FRAME_FORMAT_NV12’
         uvcFormat = UVC_FRAME_FORMAT_Y16;
                     ^~~~~~~~~~~~~~~~~~~~
                     UVC_FRAME_FORMAT_NV12
Makefile:721: recipe for target 'obj/uvcacquisition.o' failed
make: *** [obj/uvcacquisition.o] Error 1
kekiefer commented 4 years ago

I'm not really sure what's up with that output on raspberry pi, I haven't seen anything like this before, and I'm not seeing it on my 2.0 core with the same setup as you. Does it persist after power cycling the camera?

It looks like your ubuntu build isn't building code from the master+upstream-libuvc branch, the UVC_FRAME_FORMAT_Y16 would be replaced with UVC_FRAME_FORMAT_GRAY16 in this branch.

kekiefer commented 4 years ago

Is this issue resolved?

alduxvm commented 4 years ago

yes we can close it, thanks!!

kevinm54 commented 3 years ago

What was the actual resolution? I'm still getting this error on a RPi4 after following the instruction on the wiki. It seems those haven't been updated in 3 years though so probably don't include the modifications needed for the RPi4.