greiman / SdFat

Arduino FAT16/FAT32 exFAT Library
MIT License
1.08k stars 509 forks source link

Effect of truncate() after pre-allocation #296

Open AlbertBrandl opened 3 years ago

AlbertBrandl commented 3 years ago

In the TeensySdioLogger example it is written that pre-allocation of a file can reduce delays due to the search of free clusters (I guess with the final result of speeding up the writing process in a logger). However, what happens after calling the truncate() method? Does the preallocation have any effects after the truncation? Is there any way to expand the preallocation of a file which is already written?

example of a periodic sync and truncate execution (suppose Sd, file and ringbuff correctly initialized and began):

void loop(){
     static size_t numcount = 0;

    outRingBuff.write('s');

    if (ringSize >= 10 && !stateFile.isBusy()) {
          if (10 != outRingBuff.writeOut(10)) {
            Serial.println("ERROR Write from ringBuf to file failed.");
          }
        }

    if(numcount == 20){
        outRingBuff.sync();

        stateFile.sync();
        stateFile.truncate();
    }else{
        numcount++;
    }
}
greiman commented 3 years ago

FAT16/FAT32 does not support pre-allocation. SdFat pre-allocation is a trick I use and works while the file is one large contiguous block and is open. It is not possible to expand pre-allocation.

FAT assumes no extra clusters are allocated after EOF at fileSize. exFAT allows pre-allocation but FAT and exFAT only get substantial performance enhancements if the files are contiguous so no access to the FAT occurs.

Most devices and operating systems use tricks to get performance from SD cards. SD cards only deliver high performance if huge contiguous transfers are used. The actual SD flash pages are many kB, not 512 bytes so 512 byte sectors are emulated and huge overhead occurs if transfers are less than a flash page.

The RingBuf class is not really useful unless the buffer is a large multiple of 512 bytes and writeOut is called with a multiple of 512 bytes.

greiman commented 3 years ago

The main purpose of RingBuf is to avoid SD busy time. An SD card only guarantees a max write of 512 bytes when is indicates not busy. An SD card can be busy for 200 ms or more if it is moving data for wear-leveling or erasing flash to provide free blocks.