FIle format

[majo551]

Hi, first of all I would like to check if the text in README regarding file storing and reading is correct. The text says:

The following example showed how to store file data to Flash memory at "/test/file_data".

if (Firebase.setFile(firebaseData, StorateType::SPIFFS, "/test/file_data", "/test.txt"))

If I read this correctly then this is not correct, because setFile stores the file in Firebase not to SPIFFS.. I think there should be FirebasegetFile in the example.

Anyway, my question is if you can share/advise how to grab the file String encoded in the library specific 64BASE format. I need to grab the file from the Firebase node in an Android app, then decode and extract the saved jpg. For this I need to know where actually the image data start in the encoded string and how to turn them back to the original jpg file before encoding to BASE64. I am not asking to show me how to decode , what I need is to know if I can strip out the jpg part of the stream (e.g. take the data from the second slash???) and use any available BASE64 encoders to recreate the original jpg file? Thanks Marian

[mobizt]

Thank you for the readme typo, and I will correct it in the next update.

The library will make a modified base64 encoded string and store in Firebase node. When you call setFile, the library will prepend the base64 encoded string with file,base64,, and when you call setBlob, the base64 string will prepend with blob,base64,.

You can skip the first 12 bytes of the modified base64 string in Firebase node prior to decoded to the original data in your Android app.

[majo551]

Thanks this worked fine. Do you think it would make sense to request an option to save file to FB from a byte buffer ? It takes a lot of time to store the buffer to SPIFFS and then again to read it from SPIFFS when uploading the data to FB using the existing setFile method. It would be much faster I believe if I can directly push the file from a buffer. In my scenario I capture an image and store it to SPIFFS from the frame buffer and then save it to FB as a file. This takes around 5 long seconds from the time the frame is captured till stored to FB due to the slow SPIFF write/read performance Regards, Marian.

[mobizt]

Please try Firebase.setBlob function.

[mobizt]

The most time usage of the setFile function is from the chunks data sending (512 bytes each) which data will send multiple times. This is the limitation of the low memory device.

For large data sending of the setBlob function, you need to edit this https://github.com/mobizt/Firebase-ESP32/blob/master/src/FirebaseESP32.h#L2952

And be aware of setBlob uses the dynamic memory to keep the string and send it once which may exceed the transmit buffer of the WiFiClient.

[majo551]

Hi, the Blob function limitation makes sense and from this reason the function cannot be used for storing raw image bytes. The frame buffer sends usually data of size around 100 k which I am sure will be too much for the chip. I just tested the function with limited 1024 size, the speed was much larger but of course I could not upload whole image ... So a similar function like setFile for byte data that uploads whole buffer in repeated chunks would be needed in such scenario.

[mobizt]

This can't achieve between speed and size.

The setFile function design is for storing and restoring large size data instead of speed.

Due to default rx/tx buffer size in the SDK, I can't provide the option of setFile to allow the user to send large data at once.

The default mbedTLS rx and tx buffer size are 16384 bytes by default. If you need to increase it, you can set the config in the mbedTLS SDK part.

There are many discussions on this of similar issues about storing the large data in Firebase which you can search in.

In addition, Firebase RTDB is just a JSON text file database instead of a binary supported rational database like SQL, then add the large string to someplace in the text file is inefficient by default.

[majo551]

Understand, my question was not to create setFile to save all data at once but adjust setBlob (or a new function) to upload the large byte arrays string in smaller chunks the same way setFile does. It would send the blob from byte buffer but the string would be sent in small chunks . It will be still quicker than storing the data to SPIFFS and then reading it back. Thanks

[mobizt]

The setBlob takes the byte data (ram) directly, nothing is related to flash mem. Ok, I get it and may add support to the large blob data in the next update.

As I post above the time is network-dependent, flash memory reading time is small.

[majo551]

Great this would help. It would allow speeding the upload if uploaded directly from ram. Of course depending on network speed.

[mobizt]

This is for information.

The noDelay option in the HTTPClient library sends large size payload data instead of frequently small size payload and this improves the overall speed.