Lepton interface with Intel Galileo (Packet discontinous)

[subashnus]

Hi, I have used the Arduino code given at https://github.com/PureEngineering/LeptonModule for Intel's Galileo board with small modification. The modification are as follows,

void setup() { Wire.begin(); Serial.begin(115200); pinMode(10, OUTPUT); SPI.begin(); Should be initialized then appropriate parameters has to be SET SPI.setDataMode(SPI_MODE3); SPI.setClockDivider(SPI_CLOCK_DIV2); Serial.println("setup complete"); } unsigned int lepton_image[60][82]; Declared to plot the image void read_lepton_frame(void) { int i; for (int frame_number = 0; frame_number < 60; frame_number++) { for (i = 0; i < (VOSPI_FRAME_SIZE / 2); i++) { digitalWrite(10, LOW); lepton_frame_packet[2 * i] = SPI.transfer(0x00); lepton_frame_packet[2 * i + 1] = SPI.transfer(0x00); lepton_image[frame_number][i] = (lepton_frame_packet[2_i] << 8 | lepton_frame_packet[2_i+1]);
digitalWrite(10, HIGH); } lepton_image_index[frame_number]=lepton_frame_packet[0]; Trying to get the index of each packet in a frame } } void loop() { int i; int frame_number; while (1) { lepton_sync(); read_lepton_frame(); for (int frame_number = 0; frame_number < 60; frame_number++) { Serial.println(lepton_image_index[frame_number]&0x0f); printing the Packet index to verify the data capture is in sync } } }

I am basically trying to capture the packet index of each packet in a frame and saving it in an array to render the Thermal image. The code complies and gets downloaded into the Galileo baord and when i watch the serial port the packet index is not continuous it is discontinuous like,

0 0 0 0 0 12 13 0 0 0 0 1 2 0 0 0 instead of 1 2 3 4 5 6 7 8 ...... 60.

i tried changing the SPI clock speed by varying the the parameter in SPI.setClockDivider as well as the delay time of 185ms for establishing the sync (as given in the datasheet). No matter what i do the packets are discontinuous. It will be really helpful if anybody aware of the issue i am facing. If anybody is successful is getting the thermal image using the code given for Arduino , Kinldy let me know the modifications that i need to do.

Thanks, Subash

[tz1]

It won't work unless you read 164 bytes together each cycle. and the first few might be stored, so don't just get sixty, loop and store by the row returned until you get row 59, and don't do anything between. the timing can be tight

[subashnus]

Hi tz1, Yeah you are right. When you look at the function void read_lepton_frame(void), I am exactly doing what you have said. I am reading all the 164 bytes then I am going to next row. Correct me if I am wrong.

Thanks, subash

[tz1]

It needs to be CS low, 164 bytes, cs high. You do 82 ( cslow 2 bytes cshigh )

That is in a for (framenum=0..60).

You need: Do { Transfer frame to buffer; check valid and frame (line) number; If valid frame num Then Copy buffer to frame-line in image; } until frame 59 received.

See the 12/17 posts from https://groups.google.com/forum/m/#!topic/flir-lepton/MCy66OHf0l4 which contains a sketch that does it this way. I don't think I pushed them to the tz1 branch here yet. On Dec 30, 2014 5:54 AM, "subashnus" notifications@github.com wrote:

Hi tz1, Yeah you are right. When you look at the function void read_lepton_frame(void), I am exactly doing what you have said. I am reading all the 164 bytes then I am going to next row. Correct me if I am wrong.

Thanks, subash

— Reply to this email directly or view it on GitHub https://github.com/PureEngineering/LeptonModule/issues/9#issuecomment-68354115 .

[subashnus]

Hi tz1, I have checked the code in the given link, it is same CS low, counter size 82 with 2 byte increment, CS high. Does it different from, CS low, counter size164 with 1 byte increment, CS high. I even tried using the code given in the link that you have suggested, but I didn't get any frame index, it just polls in a loop. Could you post the updated code in Github ?. As well as I observed that, when i use SPI.setClockDivider(SPI_CLOCK_DIV2 ) for Intel Galileo it is faster when compared to SPI.setClockDivider(0);

Thanks, Subash

[tz1]

There is an upper limit on the SPI clock frequency, try 10-20 MHz. This is from your original message:

for (i = 0; i < (VOSPI_FRAME_SIZE / 2); i++) { digitalWrite(10, LOW); lepton_frame_packet[2 * i] = SPI.transfer(0x00); lepton_frame_packet[2 * i + 1] = SPI.transfer(0x00); lepton_image[frame_number][i] = (lepton_frame_packet[2i] << 8 | lepton_frame_packet[2i+1]); digitalWrite(10, HIGH); }

The above code lowers CS, transfers 2 bytes, then raises it. The digitalWrite() calls are INSIDE the loop in your code. The equivalent are on the outside in mine.

So at this point I don't understand what you are saying. Is the code you posted what you are doing or is the description above? Also, my SPI code is optimized to read as fast as possible. Having a fast SCI clock doesn't help if you have to wait 100 microseconds doing other things between reading the bytes. There may also be hardware problems.

The latest code is equivalent to that in the link, I'm on vacation but will see if I can test the code.and update something here.

I would suggest writing a very fast SPI read core that can drop the CS, transfer 164 bytes, raise the CS, and see if you get valid data frames. Then store them in an image buffer based on the row number in the data frame, and do your processing when the row counter reaches 59.

[subashnus]

Hi tz1, I have modified the code as per your suggestions, the changes that I have done are as follows, void read_lepton_frame(void) { int i; image_index[0]={0}; //Global variable initialized inside the function digitalWrite(10, HIGH); digitalWrite(10, LOW); for (i = 0; i < (VOSPI_FRAME_SIZE); i++) { lepton_frame_packet[i] = SPI.transfer(0x00); } digitalWrite(10, HIGH); if(((lepton_frame_packet[0]&0xf) != 0x0f)) { frame_number = lepton_frame_packet[1]; if(frame_number < 60 ) { image_index[frame_number]=frame_number;
lost_frame_counter = 0; for(i=0;i<80;i++) { lepton_image[frame_number][i] = (lepton_frame_packet[2_i+4] << 8 | lepton_frame_packet[2_i+5]); } } } else { void lepton_sync(void); } } void lepton_sync(void) { int i; int data = 0x0f; digitalWrite(10, HIGH); delay(185); while (data & 0x0f == 0x0f) { digitalWrite(10, LOW); data = SPI.transfer(0x00) << 8; data |= SPI.transfer(0x00); digitalWrite(10, HIGH); //PORTB |= 1 << 2; digitalWrite(10, LOW); for (i = 0; i < ((VOSPI_FRAME_SIZE - 2) / 2); i++) { SPI.transfer(0x00); SPI.transfer(0x00); } digitalWrite(10, HIGH); } }

Now the code works but takes about a minute to get all the frame index. Initially, few frame indexes are pushed to the serial port. Gradually all the frame indexes are coming. Is this the general scenario for Lepton frame synchronization ? or am I doing something wrong ?. Sorry for disturbing you during your vacation. After vacation, kindly update the code in Github, Meanwhile if I get any results I will post it here. Advanced new year wishes...

Thanks, Subash

[tz1]

Remove the lepton_sync call. If you get something out of sync, you are probably confusing things. Just put a the delay of 185 after the initial digitalWrite(10,HIGH) before getting frames. (No problem with my vacation, I don't mind answering, if I did, I'd wait). Also, you should fix the endian problem (the 2*i+4]<<8) after the copy. Just memcpy 160 bytes from the buffer (starting at byte 4 - note the index of byte 4 is [3] since it starts at zero) to the image array and then do the swap after you have a complete image.

[subashnus]

Hi tz1, I have tried removing void lepton_sync(void) and putting the delay as you said. It does not give the frame indexes as expected. The lepton_sync(void) function waits for 5 frames and only returns if the frame index is a number. Hence, if I put lepton_sync(void) it yields exact frame indexes in a minute, after running in a loop. However, the packets will be from different frame. If I don't use two loops, one for row and other for each packet, the packets collected will be from different frame. I have tried different methods as given in raspberrypi_capture, stm32nucleo_401re examples but nothing is giving accurate frame indexes to render the image. Only in raspberrypi_video example, the LeptonThread.cpp ensures that all the packets are from same frame as given below, while(true) { int resets = 0; for(int j=0;j<PACKETS_PER_FRAME;j++) { read(spi_cs0_fd, result+sizeof(uint8_t)_PACKET_SIZE_j, sizeof(uint8_t)_PACKET_SIZE); int packetNumber = result[j_PACKET_SIZE+1]; if(packetNumber != j) { j = -1; resets += 1; usleep(1000); if(resets == 750) { SpiClosePort(0); usleep(750000); SpiOpenPort(0); } } } I even tried like the above code with necessary changes, but it simply polls in Galileo. May be if you run the code once in Galileo, you will get a clear picture of the issue I am facing.

Thanks, Subash

[adgriff2]

I've never used Galileo, but what you're describing happens in arduino also. I had to extensively modify the arduino code to get sequential packet numbers.

Basically, I wasn't pulling data from the Lepton fast enough. The lepton exports frames at a rate of 27Hz. (These are sequences of 3 identical frames to comply with US IR export restriction of 9Hz video.) That means that you need to be reading 60 packets in less than 1/27 seconds or you'll never get synced. It will send out garbage instead. (mostly 0's with groupings of non-zeros) If you're using the 164 length packet, this would be (164 * 8 * 60) / (1 / 27) = 2125440 b/s, or SPI speed of at least 2.027 MHz, assuming zero downtime between packets. (Not sure if Galileo has an SPI buffer.) Arduino has a very large delay between SPI packets so you're effective data rate is well below what you would calculate from SPI.setClockDivider. I ended up using an external 20MHz crystal on my arduino.

I'd recommend going through a simple loop at max speed reading packets and only storing the 2nd byte (index) into a very long vector. I'd expect the vector to look one of 2 ways.

First the bad vector: 0 0 0 0 0 garbage garbage garbage 0 0 0 0 ...

Or the good vector: 0 0 0 0 0 0 0 1 2 ... 58 59 0 0 0 0 0 0 0 0 1 2 ... 58 59 0 0 0 0

Something like

int index[100000]

for (int i=0; i<100000; i++){
  for (int j=0; j<164; j++){
    if (j==1){
      index[i]=readSPI;
    }else{
      readSPI;
    }
  }
}

[subashnus]

Hi Adam Griffin, I have checked as you said. I am not getting a continuous packet. Sometimes it is continuous with some intermediate values like 1 175 2 175 3 175 4 175 5 175 6. I don't know what is that value 175 between the packet index. This result is for a SPI clock of 12.5Mhz with Galileo.

Thanks, Subash

[tz1]

175 = 0xaf = busy/wait. If you are faster than it can acquire data, it will get these. Throw away (ignore, don't reset) any packet that is not between 0 and 59.

[pale1805]

Hello Everyone!

I am trying to connect the Flir Lepton Ir camera to an Intel Galileo but I cannot get the code on the pure engineering webpage or the one on this forum to work! Could anyone please help me? Does anyone have the complete functional code? Thanks!!

[PureEngineering]

How are you connecting the two together. the intel is 1.8V logic. I don't have a intel galileo, so I haven't tried it myself yet. I think you need level translators on the SPI lines, and I'm guessing just pull-up resistors on the i2c to 2.8V.

[pale1805]

Hello!

I was finally able to connect ever thin in place. I had to use an Arduino DUE (84Mhz clock), but I am finally reading output from the camera. However, I am stuck getting always the first frame/packet. This is the seri a output (packet values separated by '' for illustrations purposes:

'0','1','4A','66','1A','45','1A','7E','1A','A4','1A','B9'….. '0','1','DC','63','1A','46','1A','7D','1A','9F','1A','B4'…. '0','1','61','3D','1A','39','1A','7B','1A','9A','1A','B3'…

can anyone help me understand what is going on here? why I can get the second packet?

[PureEngineering]

If you are printing out the serial port, you block the cpu from reading out the next packet. you should capture out an entire image, then print out your data.
sending out the serial port will cause you to lose sync with the lepton. if you can bump up the baud rate to 921600, then you should be able to get higher framerate out through the serial port.