Send image over http request

[Fernando-Castilho]

Hi, I found out the FPM library two weeks ago, I used it to store fingerprints on my pc using the python example, after that I used some C# code to compare the generated images and it worked. But I need to send the image to my web server that is in laravel (php framework) and has a path just for the image upload, I can upload the file using a web form, but I don't figured out how I can create a post request using the Arduino with the fingerprint image, I am using the ENC28J60 module, so I can do post requests using UIPEthernet library

The following code is used to do a post request with a text message:

client.println("POST /test HTTP/1.1");
client.println("Host: 192.168.1.104");
client.println("Content-Type: multipart/form-data");
client.println("Connection: close");
client.print("Content-Length: ");
client.println(message.length());
client.println();
client.print(message);
client.println();

If I understood correctly, the image_to_udp example has the function readRaw and the buffer variable is where the image is stored, but how can I send it over the post request?

finger.readRaw(FPM_OUTPUT_TO_BUFFER, buffer, &read_finished, &readlen);

Thanks for the help!

[brianrho]

Hi, yes, each call to readRaw() gives you a 128-byte chunk of the image data -- overall, there should be around 37 kB, assuming the image is 256 x 288 pixels, with each byte representing the colour information for 2 adjacent pixels.

That example simply crafts a UDP packet for each 128-byte chunk and sends it off to some server, which will . It's not the most practical example, since in real usage, you'd probably want to send some sort of "packet header" to the server first, to tell it that you're about to send it an image, so that it gets ready to parse, assemble and store the image from the chunks.

That's actually what you'll be doing here with a POST. The image data is binary (not text) data, which should go in your POST request's body, and your POST headers should inform your server -- mainly by means of the URI and Content-Type -- exactly what data is contained in the body, and how to interpret/use/store it, as applicable.

So, to POST this binary data, you'll likely need to use the Content-Type: multipart/form-data to setup the start of the POST request and then in subsequent subparts, the Content-Type can be application/octet-stream, in the while loop as you call readRaw(). Your server reads each chunk, and assembles a BMP header + image from those chunks, as shown here: https://github.com/brianrho/FPM/blob/master/extras/GetImage/getImage.py.

This page seems to explain the HTTP Multipart pretty well, especially with its linked pages/references.

[brianrho]

You can also see an example of Arduino HTTP Multipart here, sending a file: https://github.com/nailujx86/ESP8266_multipart/blob/master/ESP8266_multipart.cpp.

(Slightly buggy since it seems it calculates the Content-Length incorrectly, and is missing a few CRLFs)

[brianrho]

So, something along the very-rough lines of:

client.println("POST /test HTTP/1.1");
client.println("Host: 192.168.1.104");
client.println("Content-Type: multipart/form-data; boundary=X-ARDUINO_MULTIPART\r\n");
client.println("Connection: close");
client.println("Content-Length: XXX");
client.println("\r\n\r\n--X-ARDUINO_MULTIPART");

uint16_t count = 0;

while (true) {
        /* start composing packet to remote host */        
        bool ret = finger.readRaw(FPM_OUTPUT_TO_BUFFER, buffer, &read_finished, &readlen);

        if (ret) {
            count++;

            /* we now have a complete packet, so send it */
           client.println("Content-Disposition: form-data; name=\"blah.bmp\"");
           client.print("Content-Type: application/octet-stream\r\n\r\n");
           client.write(buffer, readlen);

            /* indicate the length to be read next time like before */
        readlen = TRANSFER_SZ;
            if (read_finished)
            {
                client.print("\r\n--X-ARDUINO_MULTIPART--\r\n\r\n");
                client.flush();
                break;
            }

           client.print("\r\n--X-ARDUINO_MULTIPART\r\n");
           client.flush();
        }

[brianrho]

I'll be closing this, if this answers your questions. Maybe I'll update the example at some point, to send the image over HTTP, rather than just raw UDP datagrams.

[Fernando-Castilho]

Hi, sorry for the late response, I couldn't use the code because arduino uno didn't had enough memory, I purchased a mega 2560 and now I have enough hardware. I managed to test the code today, but I am getting two errors, sometimes it returns wrong read length: -1 or packet to long: 130. I tested again the image_to_pc and it worked fine, so probably it's my fault. This is the code:

void GetFingerprint(){
  if (!set_packet_len_128()) {
    Serial.println("Could not set packet length");
    return;
  }

  delay(100);

  int16_t p = -1;
  Serial.println("Waiting for a finger...");
  while (p != FPM_OK) {
    p = finger.getImage();
    switch (p) {
      case FPM_OK:
        Serial.println("Image taken");
        break;
      case FPM_NOFINGER:
        break;
      case FPM_PACKETRECIEVEERR:
        Serial.println("Communication error");
        break;
      case FPM_IMAGEFAIL:
        Serial.println("Imaging error");
        break;
      default:
        Serial.println("Unknown error");
        break;
    }
    yield();
  }

  p = finger.downImage();
  switch (p) {
    case FPM_OK:
      Serial.println("Starting image stream...");
      break;
    case FPM_PACKETRECIEVEERR:
      Serial.println("Communication error");
      return;
    case FPM_UPLOADFAIL:
      Serial.println("Cannot transfer the image");
      return;
  }

  if (client.connect(server, port)){
    Serial.println("Connected to server");
    client.println("POST /post HTTP/1.1");
    client.println("Host: 192.168.1.106");
    client.println("Content-Type: multipart/form-data; boundary=X-ARDUINO_MULTIPART\r\n");
    client.println("Connection: close");
    client.println("Content-Length: 36864");
    client.println("\r\n\r\n--X-ARDUINO_MULTIPART");

    bool read_finished;
    uint16_t readlen = TRANSFER_SZ;
    uint16_t count = 0;

    while(true){
      bool ret = finger.readRaw(FPM_OUTPUT_TO_BUFFER, buffer, &read_finished, &readlen);
      if (ret) {
        count++;

        /* we now have a complete packet, so send it */
        client.println("Content-Disposition: form-data; name=\"blah.bmp\"");
        client.print("Content-Type: application/octet-stream\r\n\r\n");
        client.write(buffer, readlen);

        /* indicate the length to be read next time like before */
        readlen = TRANSFER_SZ;
        if (read_finished)
        {
            client.print("\r\n--X-ARDUINO_MULTIPART--\r\n\r\n");
            client.flush();
            break;
        }

        client.print("\r\n--X-ARDUINO_MULTIPART\r\n");
        client.flush();
      }
    }
  }

  while(client.connected()) {
    while(client.available()){
      char c = client.read();
      Serial.print(c);
    }
  }
}

I don't know why but the errors can change between the tests, also, thanks for the help before.

[brianrho]

Ah, that's probably happening because your HTTP client isn't pushing the data out quickly enough. You end up losing image data from the sensor, as the UART buffer gets filled and the UART driver starts discarding data. I'd forgotten about this old factor.

To confirm that's the problem:

• Remove all the client.flush() calls -- they shouldn't be here anyways, since the API doesn't mean what I thought it meant here. It's freeing all the used blocks immediately, rather than flushing all the outgoing data in the socket in blocking manner (which isn't a good idea either, for this usecase)
• Temporarily comment out all the client.* calls inside the while loop, so that you can confirm that when you subtract all network activity, the image is successfully read from the sensor everytime. That there are no errors in your serial logs.

Once you've confirmed this, we'll then have to think of a way to optimize the data flow from UART -> HTTP such that data is never lost. Perhaps the FPM_OUTPUT_TO_STREAM option will be useful here.

(It may be in the end that streaming each image chunk will not be feasible in real-time -- in which case, you may have to get a device with oodles of RAM, at least 48 K (like some STM32 or ESPxxxx) in order to be able to hold the entire image (37 kB) in RAM before sending it off at leisure over HTTP.)

[Fernando-Castilho]

• I removed all the client.flush() and the content-length from the second part, sometimes it can even read all the image, when that happens, the server responds with malformed HTTP request, but most of the times the error still occurs. One note is that the packet that happens the error most times is more than 120 now, before was less than 50.

• Commenting out all the client.* resolved the data loss issue.

Maybe we could do like on the image_to_pc sketch and pass the data through serial to the http?

[Fernando-Castilho]

Perhaps the malformed http request it's because the server await something like "message=I am a text message" and the code only sends the imagem, maybe if I do a client.print("fingerprint=") and after that the client.write(buffer, readlen) the issue will be fixed. I am not at home right now, gonna test it later.

[Fernando-Castilho]

Do you think that an ESP32 would be better for solving this?

[brianrho]

Commenting out all the client.* resolved the data loss issue.

Okay, that's good to know.

I've modified the example a bit here, to frame the HTTP request more correctly: https://gist.github.com/brianrho/e95c893de4fd980ec610ca60b5de013f

Integrate this into your sketch and see if things improve. (When testing, get a dump from your server end, of the request headers it received from the client, so we can inspect the structure if it's still malformed.)

If you still don't get the complete image with this, then we'll look into streaming directly from UART -> HTTP, if that will help.

[brianrho]

Do you think that an ESP32 would be better for solving this?

Generally, it would make things easier for you since you can read the entire image into RAM, before sending it to the server at any rate you like, no pressure.

You don't need to get an ESP32 specifically, even some STM32 or similar with loads of RAM should suffice.

But if you want, you can leave that till you've exhausted all other options.

[Fernando-Castilho]

It's almost working, the code you sent me worked, but there's two problems:

• Sometimes, it will be giving me the wrong read length error again, but this time I think I know what it is, the ethernet module probably is with some issues in some wires, so it is able to connect to the server, but on the while loop, it get disconected, I think that's the problem because some of my string message post started to do the same thing, I need to run it 10 times do get connected to the server. I found out that was the problem because I changed some wires and it worked normal, so the code isn't the problem.

• Now the second problem, when my ethernet module worked fine, it could do some post request, the http malformed error don't shows up anymore and the server receives the file, but the file type is wrong, the server is receiving a bin file, I saved it and tried to convert to a bmp, but it didn't worked. My thinking is that it is probably corrupted, as it have 36864 bytes, so it received all the image data.

I tried to change the ´char fileName[] = "finger.bmpp"´ to ´char fileName[] = "finger.bmp"´, but didn't worked, still receiving the bin file.

[brianrho]

I found out that was the problem because I changed some wires and it worked normal, so the code isn't the problem.

So after fixing your wiring issues, are you able to reliably receive all 36864 bytes of the image every time, without HTTP errors? If not everytime, what's the % rate of failures? Do these failures happen only with this image_http sketch or do they happen when you POST other stuff as well?

Also, how long does it take roughly to POST the entire image to the server? Any idea why the server disconnects in the middle and how often this happens?

[brianrho]

but the file type is wrong, the server is receiving a bin file, I saved it and tried to convert to a bmp, but it didn't worked.

So the file type is "correct", .bmpp is not a typo -- I intended it to mean "BMP part", implying that this is only a part of a BMP image, not the entire thing.

• The sensor doesn't provide an entire properly-formatted image. Instead, it provides a raster -- 1-byte of grayscale-colour information for every pixel in the 256 x 288 image -- i.e. a total of 73728 bytes.
• Actually, it doesn't provide the complete colour for every pixel -- rather each byte contains information for 2 pixels. That is, for each byte sent by the sensor, the high-nibble of that byte contains the high-nibble-colour of one pixel, while the low-nibble of that same byte contains the low-nibble-colour of the adjacent pixel. That is, something like so:

(Though typically, pixels that are that close to each other will have very similar colours and thus similar high-nibbles. So, more likely, the 2 pixel colours above should've been 0xAB and 0xAD.)

Quoting the poorly-translated ZFM20 datasheet:

Upload or download images through UART port in order to speed things up, only use pixel high four bytes, ie 16 gray Each byte represents two pixels (high nibble of one pixel, the lower four bits of one pixel of the next adjacent column in the same row, ... Since the image of 16 gradations, uploaded to the PC display (corresponding BMP format), Gray scale should be extended (extended to 256 levels of gray, 8bit bit bitmap format).

This means you actually receive 73728 / 2 = 36864 bytes from the sensor -- as you've observed.

It also means that before you can view the image, you need to do a bit of processing on the 36 kB raster your server received -- mainly to prepend a BMP header to it and to fill out the missing pixels. That's what you'll see happening in this Python script, which you used to stream the image to your PC over Serial: https://github.com/brianrho/FPM/blob/master/extras/GetImage/getImage.py.

If you want to view the image on your server, you'll need to do similar processing. You can try your hand at porting the relevant parts of that script, using whatever programming language your server's using.

[Fernando-Castilho]

So after fixing your wiring issues, are you able to reliably receive all 36864 bytes of the image every time, without HTTP errors? If not everytime, what's the % rate of failures? Do these failures happen only with this image_http sketch or do they happen when you POST other stuff as well?

After fixing the wiring issues, I could send the image through the POST every time, like a 100% rate or something like 95-99%, I don't remember having any issues. I already had this problem before when posting a text, the server disconnects and I don't know why, I think that's probably the wires because when I remove and plug then again, it works for like 20-30 minutes. When the wire is with malfunction, it's rather common to receive a error while trying to do the Ethernet.begin(mac).

Also, how long does it take roughly to POST the entire image to the server? Any idea why the server disconnects in the middle and how often this happens?

it's about 7-8 seconds and it's constantly.

[Fernando-Castilho]

I will be trying to understand better the Python script and port it to my php server. Thanks for explaining me how to do the code and post the data, seriously, I couldn't do anything if it weren't for your help. I think I undestood what I need to do with your explanation right now.

[Fernando-Castilho]

It worked!!! Thank you very much! I adapted your python code to receive the bin file generated by the sensor, read each byte and then generate the image, so basically, I just call the script in php, pass the file name that should be created and the path to the bin file.

If you ever want to add a image_to_http example, here is the arduino code, feel free to modify it:

#include <SPI.h>
#include <UIPEthernet.h>
#include <FPM.h>

FPM finger(&Serial1);
FPM_System_Params params;

uint8_t mac[] = { 0x54, 0x55, 0x58, 0x10, 0x00, 0x24 };
IPAddress ip[] = { 192, 168, 1, 200 };

IPAddress server(192, 168, 1, 106);
uint16_t port = 8000;

EthernetClient client;

void setup()
{
    Serial.begin(9600);
    Serial1.begin(57600);
    Serial.println("Send image using post test");

    if (finger.begin()) {
        finger.readParams(&params);
        Serial.println("Found fingerprint sensor!");
        Serial.print("Capacity: "); Serial.println(params.capacity);
        Serial.print("Packet length: "); Serial.println(FPM::packet_lengths[params.packet_len]);
    }
    else {
        Serial.println("Did not find fingerprint sensor :(");
        while (1) yield();
    }

    while(!Ethernet.begin(mac)){
      if (Ethernet.linkStatus() == LinkOFF)
      {
        Serial.println("Ethernet cable is not connected.");
        while(1);
      }
      Serial.println("DHCP error. Trying again...");
      delay(1000);
    }
    Serial.println(F("DHCP Worked"));
}

void loop() {
    post_image();
    delay(1000);
    //while (1) yield();
}

/* set to the current sensor packet length, 128 by default */
#define TRANSFER_SZ 128
uint8_t buffer[TRANSFER_SZ];

void post_image(void) {
  if (!set_packet_len_128()) {
    Serial.println("Could not set packet length");
    return;
  }

  delay(100);

  int16_t p = -1;
  Serial.println("Waiting for a finger...");
  while (p != FPM_OK) {
    p = finger.getImage();
    switch (p) {
      case FPM_OK:
        Serial.println("Image taken");
        break;
      case FPM_NOFINGER:
        break;
      case FPM_PACKETRECIEVEERR:
        Serial.println("Communication error");
        break;
      case FPM_IMAGEFAIL:
        Serial.println("Imaging error");
        break;
      default:
        Serial.println("Unknown error");
        break;
    }
    yield();
  }

  if (connectAndAssembleMultipartHeaders())
  {
    /* The client is ready to send the data. Now request the image from the sensor */
    p = finger.downImage();
    switch (p) {
    case FPM_OK:
      Serial.println("Starting image stream...");
      break;
    case FPM_PACKETRECIEVEERR:
      Serial.println("Communication error");
      return;
    case FPM_UPLOADFAIL:
      Serial.println("Cannot transfer the image");
      return;
    }

    bool read_finished;
    uint16_t readlen = TRANSFER_SZ;
    uint16_t count = 0;

    while(true)
    {
      bool ret = finger.readRaw(FPM_OUTPUT_TO_BUFFER, buffer, &read_finished, &readlen);
      if (ret) {

        count++;

        client.write(buffer, readlen);

        /* indicate the length to be read next time like before */
        readlen = TRANSFER_SZ;
        if (read_finished)
        {
          client.print("\r\n--X-ARDUINO_MULTIPART--\r\n\r\n");
          break;
        }

      }
      else {
        Serial.print("\r\nError receiving packet ");
        Serial.println(count);
        return;
      }
    }
  }

  while(client.connected()) {
    while(client.available()){
      char c = client.read();
      Serial.print(c);
    }
  }
}

/* set packet length to 128 bytes,
   no need to call this for R308 sensor */
bool set_packet_len_128(void) {
  uint8_t param = FPM_SETPARAM_PACKET_LEN; // Example
  uint8_t value = FPM_PLEN_128;
  int16_t p = finger.setParam(param, value);
  switch (p) {
    case FPM_OK:
      Serial.println("Packet length set to 128 bytes");
      break;
    case FPM_PACKETRECIEVEERR:
      Serial.println("Comms error");
      break;
    case FPM_INVALIDREG:
      Serial.println("Invalid settings!");
      break;
    default:
      Serial.println("Unknown error");
  }

  return (p == FPM_OK);
}

#define HEADER_BUF_SZ   256
char headerBuf[HEADER_BUF_SZ];

char controlName[] = "fingerprint";
char fileName[] = "finger.bmpp";

uint16_t connectAndAssembleMultipartHeaders(void)
{
    uint16_t bodyLen = 0;

    if (client.connect(server, port)) 
    {
      Serial.println("Connected to server");
      client.println("POST /post HTTP/1.1");
      client.println("Host: 192.168.1.106");
      client.println("Content-Type: multipart/form-data; boundary=X-ARDUINO_MULTIPART");
      client.println("Connection: close");

      /* Copy boundary and content disposition for the part */
      uint16_t availSpace = HEADER_BUF_SZ;
      int wouldCopy = 0;
      wouldCopy = snprintf(headerBuf + wouldCopy, availSpace, "--X-ARDUINO_MULTIPART\r\nContent-Disposition: form-data;"
                                                      " name=\"%s\"; filename=\"%s\"\r\n", controlName, fileName);

      if (wouldCopy < 0 || wouldCopy >= availSpace)
      {
        Serial.println("Header buffer too small. Stopping.");
        return 0;
      }

      bodyLen += wouldCopy;
      availSpace -= wouldCopy;

      /* Copy content type for the part */
      wouldCopy = snprintf(headerBuf + wouldCopy, availSpace, "Content-Type: application/octet-stream\r\n\r\n");

      if (wouldCopy < 0 || wouldCopy >= availSpace)
      {
        Serial.println("Header buffer too small (2). Stopping.");
        return 0;
      }

      bodyLen += wouldCopy;
      availSpace -= wouldCopy;

      /* Add the image size itself */
      uint16_t IMAGE_SZ = 36864;
      bodyLen += IMAGE_SZ;

      /* Add the length of the final boundary */
      bodyLen += strlen("\r\n--X-ARDUINO_MULTIPART--\r\n\r\n");

      /* Send content length finally -- a sum of all bytes sent from the beginning of first boundary
        * till the last byte of the last boundary */
      client.print("Content-Length: "); client.print(bodyLen); client.print("\r\n\r\n");

      /* Then send the header for the first (and only) part of this multipart request */
      client.print(headerBuf);

      return bodyLen;
    }
    else{
      Serial.println("Connection error");
    }

    return 0;
}

The Python code I basically just changed the getPrint() to read a bin file and generate a new image with it, here is the code, feel free to modify it too:

def CreateImage(filename, storagePath):
    fingerpint = open(storagePath + "\\public\\fingerprints\\" + filename + ".bmp", "wb")
    fingerpint.write(assembleHeader(WIDTH, HEIGHT, DEPTH, True))
    for i in range(256):
        fingerpint.write(i.to_bytes(1,byteorder='little') * 4)

    filename += ".bin"
    with open(storagePath + "\\fingerprint\\" + filename, "rb") as file:
        while(byte := file.read(1)):
            fingerpint.write((byte[0] & 0xf0).to_bytes(1, byteorder='little'))
            fingerpint.write(((byte[0] & 0x0f) << 4).to_bytes(1, byteorder='little'))

    os.remove(storagePath + "\\fingerprint\\" + filename)
    fingerpint.close()

Again, thank you so much, without you I wasn't going to be able to do this.

[brianrho]

Sure, glad to see you were able to port it easily :-) I'll see about integrating the image_to_http example eventually. I've been planning a major re-write of the library for a while now, just haven't had the time to run tests.

I should mention I made a tiny commit yesterday, to adjust the way the pixel colours get extrapolated to file: https://github.com/brianrho/FPM/commit/9dea6741f118c004ae7d3536ca6e4435db3d31ae

It's something I should've fixed since, though it's not a big deal really, since you still get a useful image regardless. As you can see from the commit, your code should basically be doing this instead:

        while(byte := file.read(1)):
            fingerpint.write(byte * 2)

Otherwise, good luck!

[Fernando-Castilho]

Already updated the code, thanks!