search

groupgets / LeptonModule

Code for getting started with the FLIR Lepton breakout board
https://groupgets.com/manufacturers/flir/products/flir-lepton
BSD 2-Clause "Simplified" License
317 stars 197 forks source link

Support lepton3 #66

Closed higuchi-toshio-intec closed 4 years ago

higuchi-toshio-intec commented 4 years ago

Thank you for your 'LeptonModule' repositoy.

I used your origin/master branch, but lepton2.5 hangs in few minutes. I forked from https://github.com/yyasuda/LeptonModule . And I added Lepton3 support for raspberrypi_video.

kekiefer commented 4 years ago

Thanks for sending this pull request! The changes looks very well considered, and I would like to merge if it is working for everyone using this repository. As soon as I get the chance, I'll try this out on my own setup. I will appreciate feedback from anyone else watching this repo as well.

higuchi-toshio-intec commented 4 years ago

Revert a function.(automatic scaling range detection)

I got a good suggestion.

the original automatic range adjustment feature is also convenient, especially for the first time user.

kekiefer commented 4 years ago

So first off, I like the way you put this together with the command line options to choose the Lepton type and the color maps. This is a good contribution.

It's really not usable for me though. Does this run smoothly for you? On my rpi (a new pi 4), it starts off with a few frames and then it mostly just hangs. This is with a few different SPI speeds, and I also tried to nice it all the way down to -20. To be fair, I haven't had much luck running any Lepton 3 code that is based on this raspberrypi_video code in the past either.

Hopefully there's something wrong with the way I'm running this, and that there's just a little more guidance required. Can you think of anything that I should try to make this work?

higuchi-toshio-intec commented 4 years ago

I've been testing only in the following environment. (RPi3 B+ and Raspbian-stretch) https://github.com/higuchi-toshio-intec/LeptonModule/wiki/Environment

It works like this video on my environment. It looks working not bad. https://www.youtube.com/watch?v=DWNsL8pUeFU

Maybe something is different on your rpi4 and my rpi3b+. I don't know what should I check about this. sorry.

higuchi-toshio-intec commented 4 years ago

Wolud you run this branch ? https://github.com/higuchi-toshio-intec/LeptonModule/tree/support_lepton3_logs Please check difference by git. $ git diff support_lepton3 support_lepton3_logs

I think there are some cases (failed to display the image)

but I don't know the root cause

First, please execute with '-d 5' parameter. Probably you can see 'wrong segment number' or 'zero-value'. $ sudo nice -n -20 ./raspberrypi_video -cm 1 -tl 3 -min 30000 -max 32000 -d 5

Many-many logs are output with the following command. You can see the frequency of output. $ sudo nice -n -20 ./raspberrypi_video -cm 1 -tl 3 -min 30000 -max 32000 -d 10

kekiefer commented 4 years ago

With -d 5, I see almost all 'wrong segment number continuously' errors, for hundreds of times without a frame.

It's funny, with -d 10, I will see a 4 or so 'wrong segment number 0' printouts in a row, then a 'continuously 4 times [RECOVERED]' and this cycle repeats over and over, but through it I actually end up getting a video stream.

So it would seem slowing it down with this printing is helping? Could be timing related to the different raspberry pi.

higuchi-toshio-intec commented 4 years ago

I think the commit '652c0bf' is not so bad. Probably useful. https://github.com/higuchi-toshio-intec/LeptonModule/commit/652c0bf4516cdd06e2abe0f80691e43e42d79bc7

higuchi-toshio-intec commented 4 years ago

Sometimes Lepton 3.5 is working so good. But sometimes it's working so bad.

I wanted to see all segment numbers in the frame. Then I add some temporary steps. See this branch 'support_lepton3_logs' https://github.com/higuchi-toshio-intec/LeptonModule/tree/support_lepton3_logs

Please see the differences.

$ git diff support_lepton3 support_lepton3_logs LeptonThread.cpp

I got 2 log files.

Ignore character '0' in the log file. (but, Why lepton 3.5 could not return segment number?)

higuchi-toshio-intec commented 4 years ago

Actually, I'm ready to make a next pull request. https://github.com/higuchi-toshio-intec/LeptonModule/tree/reduce_qt_dependencies/software/raspberrypi_qt

These branch 'support_lepton3' and 'reduce_qt_dependencies' have same behavior. I changed the structure of the source file.

And this branch 'reduce_qt_dependencies_simple' is most simple version.

  1. without Qt
    • Method 'LeptonAction::setPixel' does nothing.
  2. Read frame data from SPI
  3. Print segment number

https://github.com/higuchi-toshio-intec/LeptonModule/tree/reduce_qt_dependencies_simple/software/raspberrypi_qt

I execute this and got following outputs. These look right behavior.

$ ./raspberrypi_video -tl 3 -cm 1 -min 30000 -max 32000
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400
001234000012340000123400

Probably image display is a heavy process for rpi.

higuchi-toshio-intec commented 4 years ago

Please try this version.

kekiefer commented 4 years ago

Thanks for your persistence with this. This looks like a very good update, I'll try it out as soon as I can.

kekiefer commented 4 years ago

OK, I tried to run this again on my pi4 and continued to have the same problems, so @matthewnavarro helped me to run some independent tests on a few other boards, and here's what he found:

Partially Works On (runs for a few seconds and stops):
RPi4
RPi3
Running Raspbian Buster

Works on:
RPi3
Running Raspbian Stretch

Does Not Work on:
RPi4
Running Raspbian Stretch (RPi4 does not work with stretch)

So it would seem that the problems with the stopping stream are somehow OS related. I'm thinking maybe the newer version of QT is messing things up, but this is just a guess. Update: I hacked the support from here into the v4l2loopback code, but still had the same problem

This is totally consistent with what we're seeing in this thread, since you've got a pi3 running stretch.

Note that this code does work with fine with a pi4 running buster using an 80x60 lepton.

I'd like to spend a little bit of time trying to understand what the issues are with buster and the lepton 3. The code you've contributed looks great. But I'd like to hold off merging this before understanding what's going on with the newer OS.

higuchi-toshio-intec commented 4 years ago

Thank you for your information.

I installed buster on my RPi3. And I saw that my lepton stopped in a short time.

Please try with '-ss 30' option, like this. $ ./raspberrypi_video -tl 3 -cm 1 -ss 30 -min 30000 -max 32000

My lepton does not look so bad.

mwelling commented 4 years ago

Its appears that there is a bug with the RPi4 SPI controller that scales the SPI clock frequency with the main clock frequency. So when the processor goes into idle the SPI clock is halved and causes issues.

https://www.raspberrypi.org/forums/viewtopic.php?t=246164

If you run the following to keep the processor from going idle, the code works perfectly:

sudo apt-get install stress
stress --cpu 1 &
DISPLAY=:0 ./raspberrypi_video -cm 1 -tl 3 -min 30000 -max 32000

Perhaps we can turn off frequency scaling or find another fix for the SPI kernel driver that prevents this errant behavior.

https://twitter.com/QwertyEmbedded/status/1200274725311664128

kekiefer commented 4 years ago

This was a good find.

We can turn off the frequency scaling by setting the performance governor:

sudo sh -c "echo performance > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor"

I see that /sys/kernel/debug/clk/clk_summary doesn't appear to accurately reflect the changes that are happening when the cpu frequency changes

matthewnavarro commented 4 years ago

Confirmed working with PureThermal Breakout for FLIR Lepton with a Raspberry Pi4 and Lepton 3.5. On commit 2c0ef01, Raspian Buster release 10

kekiefer commented 4 years ago

Code merged, thanks a lot everyone!