search

danjulio / lepton

Code and libraries to use a FLIR Lepton Thermal Imaging Camera Module
179 stars 37 forks source link

Lepton 3.5 w/ Teensy 3.2 and VSYNC enable #4

Closed TinManAkshay closed 3 years ago

TinManAkshay commented 3 years ago

Hi,

I have been thinking to connect lepton 3.5 to Teensy 3.2 and use these libraries. Before that i had few questions as follows:

  1. Has VSYNC been enabled successfully and use it in your test programs to get the synchronization among packets.
  2. What's the max frame rate you got?

If you could give me any valuable info on vsync, frame rate , sync and so on, I'd appreciate it?

Thanks, Akshay

danjulio commented 3 years ago

Hello Akshay,

  1. Yes, I have successfully used VSYNC on several platforms including the Teensy 3.2.
  2. My Teensy 3.2 code only achieved about 4.5 fps because it split its time between reading the Lepton and updating an LCD display. I believe with appropriate use of DMA channels and scheduling the Teensy platform should be able to read the full frame rate. I have achieved full frame rate on Pi, Beaglebone (using the PRUs) and ESP32.

Regards, Dan

TinManAkshay commented 3 years ago

Hi @danjulio ,

Thanks a lot for the quick response.

  1. Speaking of ESP32, I didnt find arduino based firmware in your libraries. What software tool you used here for esp32?
  2. With teensy 3.2, my project doesnt require LCD display. I just need 160*120 pixel values to be displayed on the monitor. I believe this way I can achieve the frame rate upto 9hz. Can you please suggest me what functions/logics I have to consider to just display the pixel values and get the max frame rate?
  3. There is a project which we require lepton 3.5 to work with Pi. But your libraries are sending the data to server using socket logic. As I explained in Point 2 above, that I want to get 160*120 pixel values.

Any suggestion regarding both platforms would be appreciated...

Thanks, Akshay

danjulio commented 3 years ago

Hello Akshay,

  1. I am using the Espressif ESP32 IDF for ESP32 development (currently I am using v4.0.1): https://docs.espressif.com/projects/esp-idf/en/latest/esp32/

  2. I guess you could remove all screen related code and encode the pixel data for transmission over the serial port. You'd have to make sure the serial port could handle the data and probably up the serial port buffer sizes inside the Teensy Arduino library to accommodate larger amounts of data. Not sure how to do this however.

  3. Perhaps look at other people's Raspberry Pi code for Lepton.

Regards, Dan

TinManAkshay commented 3 years ago

Hi Dan,

Even though we could get frames out with a frame rate upto 9hz max, but if I want to send the 160120 data to other medium using UART communication (like Serial prints), for an instance, Raspberry pi 3. To my knowledge, transferring the whole 160120 frame from ESP32 to Pi would reduce the frame rate right? What's your thought on that matter?

danjulio commented 3 years ago

Here's some calculations for serial data transmission. 160 x 120 pixels = 19200. Each pixel is 16-bits of data unless the camera is operating with AGC turned on in which case the pixel data is contained in the lower 8-bits so with some processing each pixel could be 8-bits. This means you have to move 19200x2 = 38400 bytes 8.7 times per second = 334080 bytes/sec (excluding any kind of framing data so you can easily decode the stream). This would require a minimum baud rate of ~2.7 Mbit/sec which is significantly higher than the serial port is usually run. In addition because there is so much data you would have to increase the internal serial buffers to avoid a slow down. So it's probably pretty tough to output data using the serial port.

Something you might look into - but I have no experience with and can't offer any additional suggestions on - is to use the USB peripheral on a Teensy 4. You should be able to achieve a lot higher data rate at the expense of having to deal with USB on the Teensy and in whatever receives the imagery.

TinManAkshay commented 3 years ago

Hi Dan,

Yes I understand those calculations which need to be taken into account if we proceed with Serial communication.

Would the Teensy 3.2 libraries work with SAMD51 (built by Microchip tech) microcontrollers like Adafruit Feather M4 and so on???

danjulio commented 3 years ago

I'm sorry Akshay, unfortunately I cannot answer that question as I have no experience with the SAMD51 (or its arduino libraries). I would think, since they are all pretty simple libraries, the answer would be "yes" but probably with some porting needed.

TinManAkshay commented 3 years ago

Dan,

Its alright. Not a problem at all....

I was wondering if I use your teens3.2 libraries on SAMD51 MCU with some porting (as you said), then I need to print 160*120 pixel values as you copied the whole packet into Buffer using the function named "CopyPackettoBuffer(uint8_t line)". How would I print the frame using that function?

Thanks, Akshay

TinManAkshay commented 3 years ago

Hi Dan,

I have been trying to compile the teensy arduino libraries with latest i2c_t3.h (which I just downloaded) release. its giving me a lot of errors in LeptonSDKEmb32OEM.cpp which is associated with i2c_t3.h library. I believe i2c_t3 had been modified after you developed these libraries.

Your Thoughts?

danjulio commented 3 years ago

Regarding serial data. I don't think you want to modify CopyPacketToBuffer as this is a low level routine that's part of the process of getting an entire image in ProcessPacket. You want to print lepBuffer once it's been filled (where my example programs call the routines to display the data). You'll have to figure out how you want to print it out, what buffering you need and what, if any, framing you want.

Regarding your compilation errors, without the specific errors, I have no idea of what's going wrong. I just updated the github repository with a version of the LeptonSDKEmb32OEM library that uses the Wire library on platforms other than Teensy.