search

danjulio / lepton

Code and libraries to use a FLIR Lepton Thermal Imaging Camera Module
183 stars 37 forks source link

Feature Request: RTSP tCam-mini #34

Open pierre-muth opened 2 years ago

pierre-muth commented 2 years ago

Dear Dan, I recently played with the ESP-CAM module with rtsp server (even add some code for OV7725 support). However I only tweak the code within plateform.io, and I don't feel I'll be able to add rtsp server feature to your project. Do you think it is something doable ? It would be very nice to be able to link the tCam-mini to any CCTV surveillance station, VLC player and so forth, as we can do with the ESP-CAM. Thanks a lot for your hard work, the tCam-mini module and the apps are amazing ! Pierre

danjulio commented 2 years ago

Hello Pierre,

I understand. A stream like the ESP-CAM modules produce would be very useful and I have thought about it. To modify the tCam-Mini firmware for that would take the following basic additions.

  1. Add a web server. Adding the server wouldn't be too difficult, and could be based on the ESP-CAM code but the web controls and associated handlers would have to be changed to match the Lepton sensor (AGC enable, Gain, Emissivity, Selected Color Palette). And if you wanted something like a spot meter then detecting hits in the image canvas and sending back commands to the server.

  2. Add code to convert the lepton raw image to a jpeg image for streaming. The lepton's raw data would have to be converted to a bitmap (and colorized using a palette) and then the bitmap converted to a jpeg image. There are libraries that could be used to take the bitmap and create a jpeg image but my concern is that they are very compute and memory intensive [slow] and the ESP32 couldn't keep up with the Lepton's 8.8 FPS output. The ESP-CAM is able to do this because the Omnivision camera modules can directly output jpeg images and all the ESP32 has to do is package them up for transmission by the web server.

I have plenty of code to create the bitmap and I understand how to include the Espressif web server. I'm not experienced with creating the HTML/CSS/JS for the web page (and sadly little time to learn) and I don't know how to speed up the process of encoding jpeg images. If those could be overcome I would consider a future firmware release with a web server. It would be cool.

Another tCam-Mini user had the idea of writing a python RTSP server utilizing both the tCam-Mini python driver and available python libraries for creating the RTSP server. I don't know how much work this might be and it would require another computer to run the server (like a Pi).

pierre-muth commented 2 years ago

Thanks a lot for replying so rapidly ! Exactly, the jpeg has to be done on the ESP. I did that for the ESP-CAN with OV7725 (https://github.com/pierre-muth/OV7725_ESP32cam). This sensor is very sensitive, but only gives RGB565 data. It is clear 8fps is almost impossible. If I remember well I have something like 3-4 fps at 640x480. While the lepton is 160x120, as you say the color palette mapping and other transformations has to be done. I'll continue to play/learn and update here. Thanks for your help! Pierre

danjulio commented 2 years ago

Well thanks for sharing that! Very interesting. Sorry I missed the bit about the OV7725. Maybe it is feasible. Can the encoder take the 160x120 lepton image and increase it to a larger image (that most people would want to see)? Maybe I'll also try to play with this a little.