blockwise write in SPIFFS to speed things up

[DedeHai]

Testing the library I found that it is VERY slow when writing bigger structs to the database. I have a struct size of about 40 bytes and writing one entry takes about 500ms (!) to write one entry to the SPIFFS file system. Now if I change your source code so that instead of writing each byte separately it writes out the whole struct in one go by passing the pointer and the struct size to the external write function and using DBfile.write(data,datasize); to write the whole block at once it speeds up significantly, like 30x. Now it takes just 10 seconds to write the whole database with 500 entries instead of a few minutes and to search for a integer value it now takes 25ms instead of 900ms. I tried to modify the original code with overloaded writing functions but my knowledge in c++ is not good enough to make it work so it is still compatible with the current version. can you add this functionality? edit: I commited my changes to the code here: 87f312cde7069318f7836221f025cd095856bd4e

The handler functions look like this:

File userDBfile; //user database file resides in SPIFFS

void userDBwriter (unsigned long address, const uint8_t* data, unsigned int datasize) {
  userDBfile.seek(address, fs::SeekSet);
  userDBfile.write(data,datasize);
  userDBfile.flush();
}

void userDBreader (unsigned long address, uint8_t* data, unsigned int datasize) {
  userDBfile.seek(address, fs::SeekSet);
  userDBfile.read(data, datasize);
}
`

[denden4444]

Hi DedeHai

Have you found a solution or had a reply yet ? Wanting to use this (or similar) for SPIFFS Do you know of any other similar projects ? Would love to chat about it if you have some time

[DedeHai]

@denden4444 yes, this issue was solved in the last release, the library now allows for block read/write. See the pull request here: https://github.com/jwhiddon/EDB/pull/28