search

groupgets / LeptonModule

Code for getting started with the FLIR Lepton breakout board
https://groupgets.com/manufacturers/flir/products/flir-lepton
BSD 2-Clause "Simplified" License
318 stars 197 forks source link

Waiting for sink - resets #37

Open Cullpepper opened 7 years ago

Cullpepper commented 7 years ago

Hi,

I've been trying to get the v4l2lepton module running and I get the message below - any suggestions? I'm not sure what it's telling me.

pi@raspberrypi:~/LeptonModule/software/v4l2lepton $ ./v4l2lepton /dev/video0
Waiting for sink
done reading, resets:
marty-sullivan commented 7 years ago

Hi, I think you can probably ignore that message. Or is it not streaming to the loopback device?

Cullpepper commented 7 years ago

Hi Marty,

Sorry, no. I was never able to get it working.

marty-sullivan commented 7 years ago

OK, I will need to know more about your setup in order to help.

A good place to start is to see if the plain raspberrypi_video works on your Pi. This will confirm the breakout board and Lepton are correctly connected and able to stream frames.

Assuming that works, the next step is to confirm that v4l2loopback module is installed correctly and streaming frames properly. Since you aren't getting the message "Failed to open v4l2sink device" I feel like this is probably working.

After you've started v4l2lepton, use v4l2-ctl to confirm that your /dev/videoX loopback device has the proper configuration. It should be streaming 80x60 BGR frames (maybe RGB, can't remember).

I suppose another question would be, how are you trying to access the loopback device to see if the frames are streaming? An easy test would just be to use VLC media player to open the stream.

spacemuff commented 7 years ago

Hi,

having something similar. The raspberry_video works, I have the Qt window open and can perform FFC etc etc. If I run the modprobe v4l2loopback it creates the dev/videoX but if I execute ./v4l2lepton /dev/video0 I get the same error message. If I try -ctl I get nothing

Thanks in advance

s

spacemuff commented 7 years ago

OK I have collected some data, hope it helps (trying to help me...)

User Controls

                keep_format (bool)   : default=0 value=0
          sustain_framerate (bool)   : default=0 value=0
                    timeout (int)    : min=0 max=100000 step=1 default=0 value=0
           timeout_image_io (bool)   : default=0 value=0
                keep_format (bool)   : default=0 value=0
          sustain_framerate (bool)   : default=0 value=0
                    timeout (int)    : min=0 max=100000 step=1 default=0 value=0
           timeout_image_io (bool)   : default=0 value=0

Thanks in advance

marty-sullivan commented 7 years ago

Thanks for the debug info. I will hopefully have time to look at this over the weekend. I want to keep this code usable because I really think it is the best way to interface with the Lepton on RPi with the basic breakouts.

marty-sullivan commented 7 years ago

Starting from the latest Raspbian image (with GUI) on Raspberry Pi 3 model B v1.2 with the PureEngineering breakout board / Lepton 2 wired. I had keyboard and mouse connected, and I did the following steps:

  1. I ran raspi-config to enable all interfacing (i2c, spi)
  2. sudo apt-get update && sudo apt-get dist-upgrade & rebooted
  3. sudo rpi-update & rebooted
  4. I installed and ran rpi-source
  5. I built v4l2loopback from source, copied v4l2loopback.ko to /lib/modules & /lib/modules/$(uname -r)
  6. sudo depmod -a
  7. sudo modprobe v4l2loopback
  8. At this point /dev/video0 is ready
  9. I built LeptonModule/raspberrypi_capture and ran to confirm that the Lepton was wired properly and able to capture frames.
  10. I built LeptonModule/v4l2lepton
  11. ./v4l2lepton /dev/video0
  12. I receive the message Waiting for Sink and then done reading, resets: every time the shutter closes.
  13. I installed and ran VLC Media Player. I opened the video capture device, chose /dev/video0 and opened it
  14. VLC successfully showed me the frames streaming from the Lepton.

Please follow these steps and see if you are still having problems.

spacemuff commented 7 years ago

Hi,

I could make it work by following your steps (plus some fiddling around)

Plus I have noticed, that sometimes the camera needs some help by disconnecting then reconnecting the SPIO CS0 cable while ./v4l2lepton /dev/video0 is running

most of the cases it solves the stuck in waiting for sink problem

marty-sullivan commented 7 years ago

I have a feeling that the problem is similar to the "red square" problem with the raspberrypi_video program. The code is mostly the same for v4l2lepton, and I did notice that I was having problems running raspberrypi_video when I went back and checked but not with the other sample programs. Not really sure what the fix is other than playing with the wiring like you have done.

Cullpepper commented 7 years ago

Don't forget to check the physical mounting of the lepton. After endless debugging, I noticed the lepton was loosely fitted in the breakout board and would slip, causing the red square.

That might not be your problem, but it's a lot easier to check and eliminate as an issue!

Sent from my iPhone

On Jan 29, 2017, at 7:18 PM, Marty Sullivan notifications@github.com wrote:

I have a feeling that the problem is similar to the "red square" problem with the raspberrypi_video program. The code is mostly the same for v4l2lepton, and I did notice that I was having problems running raspberrypi_video when I went back and checked but not with the other sample programs. Not really sure what the fix is other than playing with the wiring like you have done.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

clint3501 commented 7 years ago

Marty's guide above in this post works. I just changed the video0 to video1 since the pi camera likes video0 with the modprobe v4l2loopback video_nr=1

vlc works with this example and is on the pi itself already

kevin192291 commented 6 years ago

Hello, I am trying to get this to work, but I keep getting "Failed to open v4l2sink device.", however if I run pyLepton everything works great. I am have tried both CS0 and CS1 on the PI Zero, I would like to eventually get this streaming data to a webpage. Any help would be greatly appreciated.

Cullpepper commented 6 years ago

If you figure this out, can you please post your configuration? I tried the same thing for a while and never got it to work. I think I failed in getting the vl42 dependencies installed correctly.

Sent from my iPhone

On Jan 20, 2018, at 11:32 AM, Kevin notifications@github.com wrote:

Hello, I am trying to get this to work, but I keep getting "Failed to open v4l2sink device.", however if I run pyLepton everything works great. I am have tried both CS0 and CS1 on the PI Zero, I would like to eventually get this streaming data to a webpage. Any help would be greatly appreciated.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

marty-sullivan commented 6 years ago

The problem is certainly due to your v4l2loopback configuration. My suggestion is to spend time trying to get v4l2loopback to work with a normal webcam first to identify the problem. If you follow the steps exactly above, it should work. Although, those steps were done on raspbian-jessie and I haven't tested on stretch, so you may want to use jessie.

davidbendell commented 6 years ago

I literally just posted here to ask for help after hundreds of hours battling to make this work on my RPi and by a stroke of coincidence I think, I have resolved the "Waiting for Sink" problem. Type this in the terminal to make it run: ./v4l2lepton -v /dev/video1 -d /dev/spidev0.0 where video1 should be changed for your own device number. Don't use spidev0.1as all the documentation and notes above tell you to do. It is 0.0. Wow, the results are incredible.

marty-sullivan commented 6 years ago

Glad you got it working!

dmoreyes commented 6 years ago

Hi everyone, does this driver work with Lepton 3 model also ? I believe it's not due to resolution differences but I would love to be wrong. Dani.

marty-sullivan commented 6 years ago

As it is right now, it only supports Lepton 2. I don't have a Lepton 3 to test, but it could be as simple as changing any of the relevant constants highlighted below to match the new resolution and recompiling v4l2lepton.

I think I remembered reading at one point that the SPI interface on Raspi might not support the bandwidth needed for the Lepton 3, but that might have either changed or maybe the new Pis have improved since back then. It's worth a shot either way!

https://github.com/groupgets/LeptonModule/blob/1bc9aa08622e9b686c2ec90b95fa3dedd09d9166/software/v4l2lepton/v4l2lepton.cpp#L25-L37

dmoreyes commented 6 years ago

Thank you for your answer, this is something I could definitely test! I can change resolution parameters to 160x120 as a start but the most tricky part is probably around the need to read and drop fake images before getting the right one. Not sure if Lepton 2 works the same way.

marty-sullivan commented 6 years ago

FYI, it looks like the solution to the SPI issue is simply increasing the buffer size in the Linux kernel. You can set this in /boot/cmdline.txt:

spidev.bufsiz=131072

adgriff2 commented 6 years ago

I haven't worked with lepton 3 on a pi since 2014 and at that time the kernel was the bottleneck for the SPI interface. There was a significant overhead for each read call. If you tried to read in a buffer that was large enough to hold a full row of pixels, you'd incur the overhead on every row and not get the full frame before the next frame refreshed. This wasn't a problem on lepton 2 variants because it was 60x80 pixels, but became an issue with 120x160 pixels. The easy solution I found was to have a large buffer and read a full segment at a time instead of row at a time.

The 60x80 image is transmitted as a single segment, but the 120x160 image is transmitted as four 30x160 segments that need to be reassembled. This segmentation difference is probably the most likely complication you would run into by trying to just change any hard-coded 60 and 80 integers to 120 and 160 respectively.

As i understand it, the pi SPI overhead was a software issue that was planned to be fixed in later kernels, so i would hope that has been released by now.

On Wed, Feb 7, 2018 at 12:43 PM, Marty Sullivan notifications@github.com wrote:

FYI, it looks like the solution to the SPI issue is simply increasing the buffer size in the Linux kernel. You can set this in /boot/cmdline.txt:

spidev.bufsiz=131072

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/groupgets/LeptonModule/issues/37#issuecomment-363850262, or mute the thread https://github.com/notifications/unsubscribe-auth/AHL-5MXuKYqEdwQtTOhhpaJuPav8D2i7ks5tSeC5gaJpZM4LDWzK .

-- Adam Griffin 704-221-1988

adgriff2 commented 6 years ago

Here is some very old code (warts and all) that pulls the lepton3 image and serves it over tcp/ip to a connecting client that I wrote in C#.

Only the lepton2 version of the client/server code is published on github here. I believe i couldn't publish lepton3 code at the time because it wasn't released yet.

#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <linux/spi/spidev.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <pthread.h>
#include <iostream>
#include <time.h>
#include <limits>
#include <arpa/inet.h>
#include <linux/i2c-dev.h>
#include "SDK/LEPTON_SDK.h"
#include "SDK/LEPTON_SYS.h"
#include "SDK/LEPTON_Types.h"
#include "SDK/LEPTON_I2C_Reg.h"

// Constants that should never change
#define HEIGHT 120
#define WIDTH 160
#define BYTES_PER_HEADER 4
#define BYTES_PER_PIXEL 2
#define SEGMENTS 4
#define SEGMENT_FACTOR 2 // sqrt(SEGMENTS)
#define SEGMENT_LINES (HEIGHT/SEGMENTS)

// I2C constants
#define I2C_ADDRESS  (0x2A)
#define AGC (0x01)
#define SYS (0x02)
#define VID (0x03)
#define OEM (0x08)

#define GET (0x00)
#define SET (0x01)
#define RUN (0x02)

#define STATUS_REG (0x02)
#define COMMANDID_REG (0x04)
#define DATALEN_REG (0x06)
#define DATA0 (0x08)

// Maximum values for array allocations
#define PACKET_SIZE (BYTES_PER_HEADER + WIDTH * BYTES_PER_PIXEL/SEGMENT_FACTOR)
#define FRAME_SIZE (HEIGHT * WIDTH)
#define SEGMENT_PACKETS (SEGMENT_LINES * SEGMENT_FACTOR)
#define SEGMENT_SIZE (PACKET_SIZE * SEGMENT_PACKETS)

using namespace std;
pthread_mutex_t lock;

void message(const char *msg){
    pthread_mutex_lock(&lock);
    time_t t = time(NULL);
    struct tm tm = *localtime(&t);
    char time_string[200];
    sprintf(time_string,"[%4d-%.2d-%.2d %.2d:%.2d:%.2d] %s\n", tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec, msg);
    cout << time_string;
    pthread_mutex_unlock(&lock);
}

static const uint8_t bits = 8, mode = SPI_MODE_3;
static const uint32_t speed = 19000000, framePeriod = 111111;
static uint16_t delay = 0;
static uint16_t frame[FRAME_SIZE];
static uint8_t allSegments[SEGMENT_SIZE*SEGMENTS];

unsigned short crc16(unsigned char *packet, unsigned char len){
    char buffer[100];
    unsigned short crc = 0;
    unsigned char i, j;
    for (i = 0; i < len; i++) {
        crc ^= (packet[i] << 8);
        for (j = 0; j < 8; j++) {
            if (crc & 0x8000) {
                crc = (crc << 1) ^ 0x1021;
            } else {
                crc <<= 1;
            }
            crc &= 0xffff;
        }
    }
    //sprintf(buffer,"CRC = %i; len = %i; %02x,%02x,%02x,%02x,%02x,%02x,%02x,%02x",crc,len,packet[0],packet[1],packet[2],packet[3],packet[4],packet[5],packet[6],packet[7]);
    //message(buffer);
    return crc;
}

int ListenSocket(int portno){
    char buffer[100];
    int sockfd_listen;
    struct sockaddr_in serv_addr;

    sockfd_listen = socket(AF_INET, SOCK_STREAM, 0);
    if (sockfd_listen == -1){
        message("Error creating socket!");
        exit(-1);
    }
    bzero((char *)&serv_addr, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_addr.s_addr = INADDR_ANY;
    serv_addr.sin_port = htons(portno);
    if (bind(sockfd_listen, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) == -1){
        message("Error binding socket!");
        exit(-1);
    }
    if (listen(sockfd_listen, 5) == -1){
        message("Error listening!");
        exit(-1);
    }
    return sockfd_listen;
}

void GetSocket(int sockfd_listen, int *newsock_image, int *newsock_cmd){
    char buffer[100];
    message("Getting Socket");
    struct sockaddr_in cli_addr;
    int clilen;
    clilen = sizeof(cli_addr);
    *newsock_image = accept(sockfd_listen, (struct sockaddr *) &cli_addr, (socklen_t*)&clilen);
    if (*newsock_image == -1) {
        message("Error accepting!");
        exit(-1);
    }
    sprintf(buffer,"Socket connected to IP %s port %i", inet_ntoa(cli_addr.sin_addr), ntohs(cli_addr.sin_port));
    message(buffer);
    sprintf(buffer,"ID %i",*newsock_image);
    message(buffer);

    *newsock_cmd = accept(sockfd_listen, (struct sockaddr *) &cli_addr, (socklen_t*)&clilen);
    if (*newsock_cmd == -1) {
        message("Error accepting!");
        exit(-1);
    }
    sprintf(buffer,"Socket connected to IP %s port %i", inet_ntoa(cli_addr.sin_addr), ntohs(cli_addr.sin_port));
    message(buffer);
    sprintf(buffer,"ID %i",*newsock_cmd);
    message(buffer);
}

void *handleLeptonRead(void *threadid){
    char buffer[20];
    message("Lepton Reading thread started");
    fflush(stdout);
    struct timespec start, finish;
    int fd, frame_position, segment_position, i, j, k, timeout, odd, frame_line;//packet_number;
    uint16_t crc;
    //uint8_t packet[PACKET_SIZE];
    uint8_t tx[PACKET_SIZE];
    float min_max_255;
    memset(tx, 0, PACKET_SIZE * sizeof(uint8_t));
    static struct spi_ioc_transfer tr;
    tr.tx_buf = (unsigned long)tx;
    //tr.rx_buf = (unsigned long)packet;
    tr.rx_buf = (unsigned long)allSegments;
    tr.len = PACKET_SIZE;
    tr.delay_usecs = delay;
    tr.speed_hz = speed;
    tr.bits_per_word = bits;

    memset(frame, 42, FRAME_SIZE * sizeof(uint16_t));

    sleep(5);
    fd = open("/dev/spidev0.0", O_RDWR);
    ioctl(fd, SPI_IOC_WR_MODE, &mode);
    ioctl(fd, SPI_IOC_WR_BITS_PER_WORD, &bits);
    ioctl(fd, SPI_IOC_WR_MAX_SPEED_HZ, &speed);

    for(;;) {

        // *** TESTING
        //int testsize = 2400;
        //char testmem[testsize*4];
        //struct timespec com_time[testsize];
        //for (i = 0; i<testsize;i++) {
        //  clock_gettime(CLOCK_MONOTONIC, &com_time[i]);
        //  ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
        //  testmem[i * 4] = packet[0];
        //  testmem[i * 4 + 1] = packet[1];
        //  testmem[i * 4 + 2] = packet[2];
        //  testmem[i * 4 + 3] = packet[3];
        //}
        //FILE* fp = fopen("test_output.txt","w");
        //for (i = 0; i<testsize;i++) {
        //  fprintf(fp,"%i,%09i,%02x,%2x,%2x,%2x\n",com_time[i].tv_sec,com_time[i].tv_nsec,testmem[i * 4],testmem[i * 4 + 1],testmem[i * 4 + 2],testmem[i * 4 + 3]);
        //}
        //fclose(fp);
        //exit(-1);
        // *** TESTING

        timeout = 0;
        clock_gettime(CLOCK_MONOTONIC, &start);

         do {
            tr.len = SEGMENT_SIZE;
            //tr.rx_buf = (unsigned long)allSegments;
            ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
            crc = (allSegments[2] << 8) | allSegments[3];
            allSegments[0] &= 0x7F;
            //packet[0] &= 0x0F;
            allSegments[2] = 0;
            allSegments[3] = 0;
            clock_gettime(CLOCK_MONOTONIC, &finish);
            if ((finish.tv_sec - start.tv_sec)  > 1)
                timeout = 1;
        } while ( ((crc != crc16(allSegments, PACKET_SIZE)) | (allSegments[1] != 0) | (crc == 0)) && (timeout == 0));
        sprintf(buffer,"%02x,%02x,%02x,%02x,%02x,%02x,%02x,%02x,%i",allSegments[0],allSegments[1],allSegments[2],allSegments[3],allSegments[4],allSegments[5],allSegments[6],allSegments[7],timeout);
        message(buffer);
        if (timeout){
            sleep(1);
            continue;
        }

        //struct timespec com_time[SEGMENTS*SEGMENT_PACKETS];
        //clock_gettime(CLOCK_MONOTONIC, &start);
        //for (;;) {
            //tr.len = SEGMENTS * SEGMENT_PACKETS * PACKET_SIZE;
            //tr.rx_buf = (unsigned long)&allSegments[PACKET_SIZE];
            //ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
            //for (i = 1; i < SEGMENTS * SEGMENT_PACKETS; i++) {
            //  tr.rx_buf = (unsigned long)&allSegments[PACKET_SIZE];
            //  ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
            //}
            //clock_gettime(CLOCK_MONOTONIC, &finish);
            //sprintf(buffer,"start:%i:%i;finish:%i:%i",start.tv_sec,start.tv_nsec,finish.tv_sec,finish.tv_nsec);
            //message(buffer);
            //for (k = 0; k < SEGMENTS; k++) {
            //  for (i = 0; i < SEGMENT_PACKETS; i++) {
            //      //for (j = 0; j < PACKET_SIZE; j++) {
            //      //  allSegments[j + i * PACKET_SIZE + k * SEGMENT_PACKETS * PACKET_SIZE] = packet[j];
            //      //}
            //      if (i == (SEGMENT_PACKETS-1) && k == (SEGMENTS-1))
            //          continue;
            //      
            //      //clock_gettime(CLOCK_MONOTONIC, &com_time[i + k*SEGMENT_PACKETS]);
            //  }
            //}

            //int segment_number=0;
            //FILE* fp = fopen("test_output.txt","w");
            int temp1,temp2;
            for (i = 0; i < SEGMENTS; i++) {
                for (j = 0; j < SEGMENT_PACKETS; j++) {
                    //packet_number = allSegments[1 + i * PACKET_SIZE];
                    //if (packet_number > 59)
                    //  continue;
                    //sprintf(buffer,"Npac=%i",packet_number);
                    //message(buffer);
                    //if (packet_number == 20) {
                    //  segment_number = (allSegments[i * PACKET_SIZE] & 0x70) >> 4;
                    //  if (segment_number > 3)
                    //      break;
                    //  //sprintf(buffer,"Nseg=%i",segment_number);
                    //  //message(buffer);
                    //}
                    if (allSegments[i*SEGMENT_PACKETS*PACKET_SIZE + j*PACKET_SIZE + 1] >59)
                        continue;
                    odd = j%2;
                    frame_line = i * SEGMENT_LINES + (j - odd)/SEGMENT_FACTOR;
                    //if (segment_number!=0) {
                    //  sprintf(buffer,"%i",segment_number);
                    //  message(buffer);
                    //}
                    temp1 = odd * WIDTH/2 + frame_line * WIDTH;
                    temp2 = i*SEGMENT_PACKETS*PACKET_SIZE + j*PACKET_SIZE;
                    for (k = 4; k < PACKET_SIZE; k+=2) {
                        frame[temp1 + (k-4)/2] = (allSegments[temp2 + k] << 8 | allSegments[temp2 + k + 1]);
                    }
                    //fprintf(fp,"%02x,%02x,%02x,%02x,%02x,%02x\n",allSegments[i*PACKET_SIZE + 0],allSegments[i*PACKET_SIZE + 1],allSegments[i*PACKET_SIZE + 2],allSegments[i*PACKET_SIZE + 3],allSegments[i*PACKET_SIZE + 4],allSegments[i*PACKET_SIZE + 5]);
                }
            }
            //tr.len = PACKET_SIZE;
            //tr.rx_buf = (unsigned long)allSegments;
            //ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
        //}
        //fclose(fp);
        //exit(-1);
    }
    close(fd);
}

void *handleImageSocket(void *vptr_sockfd){
    int sockfd = *((int *)vptr_sockfd);
    char buffer[100];
    sprintf(buffer,"Image socket thread started on ID %i",(int)sockfd);
    message(buffer);
    int n;
    struct timespec start, finish;
    long sleep_duration;

    for(;;) {
        clock_gettime(CLOCK_MONOTONIC, &start);
        n = send(sockfd, frame, FRAME_SIZE * BYTES_PER_PIXEL, MSG_NOSIGNAL);
        clock_gettime(CLOCK_MONOTONIC, &finish);
        sleep_duration = framePeriod - ((long)(finish.tv_sec - start.tv_sec) * 1.0e6 + (long)(finish.tv_nsec - start.tv_nsec)/1000.0);
        if (sleep_duration > 0) {
            usleep(sleep_duration);
        }
        if (n<1){
            message("Image socket thread exiting");
            break;
        }
    }
}

void *handleCMDSocket(void *vptr_sockfd){
    int sockfd = *((int *)vptr_sockfd);
    uint8_t cmdbuffer[1000];
    int n;
    char buffer[100];

    sprintf(buffer,"Command socket thread started on ID %i",(int)sockfd);
    message(buffer);
    for(;;) {
        memset(cmdbuffer,0,sizeof(cmdbuffer));
        if (read(sockfd, &cmdbuffer, sizeof(uint8_t)*sizeof(cmdbuffer)) > 0) {
            sprintf(buffer,"Command read: %i",cmdbuffer[0]);
            message(buffer);
            sprintf(buffer,"Length read: %i",cmdbuffer[1]);
            message(buffer);
            LEP_CAMERA_PORT_DESC_T _port;
            LEP_RESULT result;
            pthread_mutex_lock(&lock);
            switch (cmdbuffer[0]) {
            case 1:

                break;
            case 2:

                break;
            case 3:

                break;
            case 4:

                break;
            case 5:

                break;
            case 6:

                break;
            case 7:

                break;
            case 8:

                break;
            case 9:

                break;
            case 10:

                break;
            case 11:

                break;
            case 12:

                break;
            case 13:

                break;
            case 14:

                break;
            case 15:

                break;
            case 16:

                break;
            case 17:

                break;
            case 18:

                break;
            case 19:

                break;
            case 20:

                break;
            case 21:

                break;
            case 22:

                break;
            case 23:

                break;
            case 24:

                break;
            case 25:

                break;
            case 26:

                break;
            case 27:

                break;
            case 28:

                break;
            case 29:

                break;
            case 30:

                break;
            case 31:

                break;
            case 32:

                break;
            case 33:

                break;
            case 34:

                break;
            case 35:

                break;
            case 36:

                break;
            case 37:

                break;
            case 38:

                break;
            case 39:
                result = LEP_OpenPort(1, LEP_CCI_TWI, 400, &_port);
                sprintf(buffer,"Opening result = %i",result);
                message(buffer);

                result = LEP_RunSysFFCNormalization(&_port);
                sprintf(buffer,"LEP_RunSysFFCNormalization result = %i",result);
                message(buffer);
                n = send(sockfd, (void*)&result, 1, MSG_NOSIGNAL);
                if (n==-1){
                    sprintf(buffer,"Send failed: %s",strerror(errno));
                    message(buffer);
                }

                result = LEP_ClosePort(&_port);
                sprintf(buffer,"Closing result = %i",result);
                message(buffer);
                break;
            case 40:

                break;
            case 41:

                break;
            case 42:

                break;
            case 43:

                break;
            case 44:

                break;
            case 45:

                break;
            case 46:

                break;
            case 47:

                break;
            case 48:

                break;
            case 49:

                break;
            case 50:

                break;
            case 51:

                break;
            case 52:

                break;
            case 53:

                break;
            default:
                message("Command Not Recognized!");
                break;
            }
            pthread_mutex_unlock(&lock);
        }else{
            message("Command socket thread exiting");
            break;
        }
    }
}

int main(int argc, char *argv[]){
    pthread_t LeptonRead_thread, ImageSocket_thread, CMDSocket_thread;
    pthread_mutex_init(&lock, NULL);
    message("Starting LeptonPi2 Server!");
    pthread_create(&LeptonRead_thread, NULL, handleLeptonRead, (void *)1);
    int sockfd_listen = ListenSocket(5000);
    int newsockfd_image, newsockfd_cmd;
    char buffer[500];
    sprintf(buffer,"HEIGHT: %i\nWIDTH: %i\nBYTES_PER_HEADER: %i\nBYTES_PER_PIXEL: %i\nSEGMENTS: %i\nSEGMENT_FACTOR: %i\nSEGMENT_LINES: %i\nPACKET_SIZE: %i\nFRAME_SIZE: %i\nSEGMENT_PACKETS: %i\nEGMENT_SIZE: %i\n",HEIGHT,WIDTH,BYTES_PER_HEADER,BYTES_PER_PIXEL,SEGMENTS,SEGMENT_FACTOR,SEGMENT_LINES,PACKET_SIZE,FRAME_SIZE,SEGMENT_PACKETS,SEGMENT_SIZE);
    message(buffer);
    for (;;) {
        GetSocket(sockfd_listen, &newsockfd_image,&newsockfd_cmd);  
        pthread_create(&ImageSocket_thread, NULL, handleImageSocket, &newsockfd_image);
        pthread_create(&CMDSocket_thread, NULL, handleCMDSocket, &newsockfd_cmd);
    }
    pthread_join(LeptonRead_thread, NULL);
    message("Lepton Read Exiting");
    exit(-1);
}
dmoreyes commented 6 years ago

Thank you all for your answers ! It really helps ! Just to give a piece of context, at the moment I'm using pylepton Python package. To make it working fine on a quite loaded RPi, I had to dedicate an entire core for my Lepton 3 image acquisition program to avoid being interrupted by other processes. Otherwise I'm experiencing plenty of de-sync, which blocks the Lepton after 10 minutes. That's the reason why I'd like to switch to C/C++ and maybe to preempt-rt kernel to avoid core dedication. Just curious, have you tried this driver under heavily loaded RPi ? If says so did you have to dedicate a core to your application (might be difficult for a driver by the way...) ?

Cullpepper commented 6 years ago

Hi- I'm out of my depth for a lot of this, but I wanted to ask what you're using to interface with your pi- (virtual desktop, hmdi plug, etc.) on such a small device, the overhead from different configs makes a real difference!

Sent from my iPhone

On Feb 7, 2018, at 5:04 PM, dmoreyes notifications@github.com wrote:

Thank you all for your answers ! It really helps ! Just to give a piece of context, at the moment I'm using pylepton Python package. To make it working fine on a quite loaded RPi, I had to dedicate an entire core for my Lepton 3 image acquisition program to avoid being interrupted by other processes. Otherwise I'm experiencing plenty of de-sync, which blocks the Lepton after 10 minutes. That's the reason why I'd like to switch to C/C++ and maybe to preempt-rt kernel to avoid core dedication. Just curious, have you tried this driver under heavily loaded RPi ? If says so did you have to dedicate a core to your application (might be difficult for a driver by the way...) ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

marty-sullivan commented 6 years ago

Sure thing, and yes thanks for your code @adgriff2

I haven't done any real performance tests, but I know that I've taken a look at htop while testing v4l2lepton. I recall there being constant CPU usage while it's streaming frames, but it did not take up a full core. My mind is pointing to about 20% of one core, but I don't trust my memory well enough to guarantee that :)

Of course, you will just have to test yourself, and this tax on the CPU will likely increase with the Lepton 3 considering it has 4x the pixels.

EDIT: Actually, now that I think about it, since I used VLC to test, the CPU usage was almost certainly from displaying the frames on screen. Perhaps there is little to no CPU involved for just streaming the frames! Please report back your results if you get this working.

adgriff2 commented 6 years ago

I've not experimented with reading the lepton with pi under heavy processor load, though I can imagine there would be an issue. It may be possible to reduce the CPU load of reading the lepton by only reading unique frames. The lepton output images at a rate of 27Hz to work with standard protocols, but only refreshes the images internally at 9Hz due to ITAR restrictions. Basically, unique frame is sent out 3 times to get from 9Hz to 27Hz. You really only need to read each unique frame once and sleep during the other 2 frames, but i've never bothered to do this.

On Wed, Feb 7, 2018 at 5:11 PM, Marty Sullivan notifications@github.com wrote:

Sure thing, and yes thanks for your code @adgriff2 https://github.com/adgriff2

I haven't done any real performance tests, but I know that I've taken a look at htop while testing v4l2lepton. I recall there being constant CPU usage while it's streaming frames, but it did not take up a full core. My mind is pointing to about 20% of one core, but I don't trust my memory well enough to guarantee that :)

Of course, you will just have to test yourself, and this tax on the CPU will likely increase with the Lepton 3 considering it has 4x the pixels.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/groupgets/LeptonModule/issues/37#issuecomment-363928502, or mute the thread https://github.com/notifications/unsubscribe-auth/AHL-5IkQToL1CnanVtv45s-gaSjNuhVFks5tSh-tgaJpZM4LDWzK .

-- Adam Griffin 704-221-1988

dmoreyes commented 6 years ago

I'm not really interfacing with Pi. It is just used to collect data and display results on the screen through analog video, then video signal is sent through a radio channel.

2018-02-07 23:08 GMT+01:00 Cullpepper notifications@github.com:

Hi- I'm out of my depth for a lot of this, but I wanted to ask what you're using to interface with your pi- (virtual desktop, hmdi plug, etc.) on such a small device, the overhead from different configs makes a real difference!

Sent from my iPhone

On Feb 7, 2018, at 5:04 PM, dmoreyes notifications@github.com wrote:

Thank you all for your answers ! It really helps ! Just to give a piece of context, at the moment I'm using pylepton Python package. To make it working fine on a quite loaded RPi, I had to dedicate an entire core for my Lepton 3 image acquisition program to avoid being interrupted by other processes. Otherwise I'm experiencing plenty of de-sync, which blocks the Lepton after 10 minutes. That's the reason why I'd like to switch to C/C++ and maybe to preempt-rt kernel to avoid core dedication. Just curious, have you tried this driver under heavily loaded RPi ? If says so did you have to dedicate a core to your application (might be difficult for a driver by the way...) ?

— You are receiving this because you authored the thread.

Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/groupgets/LeptonModule/issues/37#issuecomment-363927665, or mute the thread https://github.com/notifications/unsubscribe-auth/AOQG9CPPemThHwJnSLgjimLwvC81GdSqks5tSh75gaJpZM4LDWzK .

dmoreyes commented 6 years ago

Unfortunately, as per the documentation, you need to read every frame, including invalid ones, otherwise sync is lost :

"For both Lepton and Lepton 3, each unique video frame is followed by two non-unique frames which must be read out to maintain synchronization. For Lepton each unique video frame is duplicated twice. For Lepton 3 each unique frame is followed by two partial, invalid frames." https://groupgets-files.s3.amazonaws.com/lepton/Lepton-3-Engineering-Datasheet.pdf

2018-02-08 15:00 GMT+01:00 Adam Griffin notifications@github.com:

I've not experimented with reading the lepton with pi under heavy processor load, though I can imagine there would be an issue. It may be possible to reduce the CPU load of reading the lepton by only reading unique frames. The lepton output images at a rate of 27Hz to work with standard protocols, but only refreshes the images internally at 9Hz due to ITAR restrictions. Basically, unique frame is sent out 3 times to get from 9Hz to 27Hz. You really only need to read each unique frame once and sleep during the other 2 frames, but i've never bothered to do this.

On Wed, Feb 7, 2018 at 5:11 PM, Marty Sullivan notifications@github.com wrote:

Sure thing, and yes thanks for your code @adgriff2 https://github.com/adgriff2

I haven't done any real performance tests, but I know that I've taken a look at htop while testing v4l2lepton. I recall there being constant CPU usage while it's streaming frames, but it did not take up a full core. My mind is pointing to about 20% of one core, but I don't trust my memory well enough to guarantee that :)

Of course, you will just have to test yourself, and this tax on the CPU will likely increase with the Lepton 3 considering it has 4x the pixels.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/groupgets/LeptonModule/issues/37# issuecomment-363928502, or mute the thread https://github.com/notifications/unsubscribe- auth/AHL-5IkQToL1CnanVtv45s-gaSjNuhVFks5tSh-tgaJpZM4LDWzK .

-- Adam Griffin 704-221-1988 <(704)%20221-1988>

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/groupgets/LeptonModule/issues/37#issuecomment-364119555, or mute the thread https://github.com/notifications/unsubscribe-auth/AOQG9KUzns_Nyb9Vwp4pbY7wWzJtJh1xks5tSv4GgaJpZM4LDWzK .

adgriff2 commented 6 years ago

Yes, the non-unique frames need to be read. But if you have other overhead such as displaying video, re-transmitting it over TCP/IP, or encoding it as a video you can detect non-unique images and potentially avoid resource intensive post-processing. Again, I haven't looked into this. If merely the act of reading the vospi constitutes 90% of your load, these tricks won't help much at all.

On Fri, Feb 9, 2018 at 7:17 AM, dmoreyes notifications@github.com wrote:

Unfortunately, as per the documentation, you need to read every frame, including invalid ones, otherwise sync is lost :

"For both Lepton and Lepton 3, each unique video frame is followed by two non-unique frames which must be read out to maintain synchronization. For Lepton each unique video frame is duplicated twice. For Lepton 3 each unique frame is followed by two partial, invalid frames." https://groupgets-files.s3.amazonaws.com/lepton/Lepton-3- Engineering-Datasheet.pdf

2018-02-08 15:00 GMT+01:00 Adam Griffin notifications@github.com:

I've not experimented with reading the lepton with pi under heavy processor load, though I can imagine there would be an issue. It may be possible to reduce the CPU load of reading the lepton by only reading unique frames. The lepton output images at a rate of 27Hz to work with standard protocols, but only refreshes the images internally at 9Hz due to ITAR restrictions. Basically, unique frame is sent out 3 times to get from 9Hz to 27Hz. You really only need to read each unique frame once and sleep during the other 2 frames, but i've never bothered to do this.

On Wed, Feb 7, 2018 at 5:11 PM, Marty Sullivan <notifications@github.com

wrote:

Sure thing, and yes thanks for your code @adgriff2 https://github.com/adgriff2

I haven't done any real performance tests, but I know that I've taken a look at htop while testing v4l2lepton. I recall there being constant CPU usage while it's streaming frames, but it did not take up a full core. My mind is pointing to about 20% of one core, but I don't trust my memory well enough to guarantee that :)

Of course, you will just have to test yourself, and this tax on the CPU will likely increase with the Lepton 3 considering it has 4x the pixels.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/groupgets/LeptonModule/issues/37# issuecomment-363928502, or mute the thread https://github.com/notifications/unsubscribe- auth/AHL-5IkQToL1CnanVtv45s-gaSjNuhVFks5tSh-tgaJpZM4LDWzK .

-- Adam Griffin 704-221-1988 <(704)%20221-1988> <(704)%20221-1988>

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/groupgets/LeptonModule/issues/37# issuecomment-364119555, or mute the thread https://github.com/notifications/unsubscribe-auth/AOQG9KUzns_ Nyb9Vwp4pbY7wWzJtJh1xks5tSv4GgaJpZM4LDWzK .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/groupgets/LeptonModule/issues/37#issuecomment-364418668, or mute the thread https://github.com/notifications/unsubscribe-auth/AHL-5CLlcoafPBI_WvUkvVyTjzddlq1Oks5tTDdlgaJpZM4LDWzK .

-- Adam Griffin 704-221-1988

dforsberg55 commented 5 years ago

I am currently using a FLIR Lepton 2 with an Odroid XU4, I have set everything up as @marty-sullivan suggested. However, I am getting "waiting for sink", and it just hangs here endlessly. It never moves onto "done reading, resets:" and I can not see any feed on VLC.

My camera does work since I can grab single images using PyLepton. I have also managed to stitch images together to create a "slide show". The problem is that I need a video output that can be used in OpenCV, e.g. "cv2.VideoCapture(0)".