How to increase writing speed?

[incher91]

Hello,

how can I increase the overall writing speed of the SPI communication with my SD Card? I used the SdFat Library for a comparison and i could write data on my SD Card every 2000 microseconds. With the Fat16 Library I can only reach about 8000 microseconds for writing data.

This is my small Arduino sketch I used: Fat16 fastDataLogger_gitHubIssue.txt

Thanks for your help!

[greiman]

Fat16 was written over ten years ago when the biggest Arduino was a 168 with 1KB of SRAM and 16 KB of flash. I sacrificed speed for size.

Why do you call sync() after every line in loop?

This causes the current data block to be written to the SD, the directory block for the file to be read, updated and written. Next time the loop is entered, the data block must be read again.

This is a total transfer of four blocks or 2048 bytes for each line written.

[incher91]

Thanks for your fast response! I like your library because its small size :-) Do you know of any other library that has small size (~7 KB) and can write very fast (every 1 ms)?

If i'm honest, i dident really get the thing about block size and how many bytes per block and when it is a good time for calling sync()... Do you have a nice source for me, where it is all explained?

[greiman]

There are no small Arduino SD libraries that writes faster than 1 ms for every line.

No SD library writes at 1 ms every time. SD cards have occasional long latencies for flash erase and wear leveling.

The spec allows up to 150 ms. High quality SD cards usually have max latencies of about 25 ms.

Fat16 will write faster than 1 ms for most lines if you remove the sync call and explicitly close the file.

Here is a modified version of your program.

#include <Fat16.h>
#include <Fat16util.h> // use functions to print strings from flash memory

const uint8_t CHIP_SELECT = SS;  // SD chip select pin.

SdCard card;
Fat16 file;

long timeStamp;
//char logName[];
char logName[] = "WRITE00.TXT";
char dataString [] = "testtest";

void setup(void) {

  Serial.begin(115200);
  Serial.println();
  PgmPrintln("Type any character to start");
  while (!Serial.available());

  card.begin(CHIP_SELECT, SPI_FULL_SPEED);  // initialize the SD card
  Fat16::init(&card);       // initialize a FAT16 volume

  for (uint8_t i = 0; i < 100; i++) {
    logName[5] = i/10 + '0';
    logName[6] = i%10 + '0';
    // O_CREAT - create the file if it does not exist
    // O_EXCL - fail if the file exists
    // O_WRITE - open for write

   if (file.open(logName, O_CREAT | O_EXCL | O_WRITE)) break;
   } 

  PgmPrint("Writing to: ");
  Serial.println(logName);
  Serial.println();

  file.print(dataString);
  file.println();   
  file.sync();  // force allocation for first cluster.

timeStamp = micros();   

}
int n = 0;
void loop(void) {
  if (n > 100) return;

  file.print(timeStamp);
  file.println();
  //  file.sync();
  if (n++ == 100) {
    file.close();
    Serial.println(F("File closed"));
  }

  timeStamp = micros(); 
}

Here is output with an extra column giving the difference between timestamps. Most lines take about 500 micros to write. Most of that is formatting the timestamp in print. Longer times are actual write to the SD. Like the 2604 micros when a 512 sector is written to the SD.

testtest    
9081820 
9083672 1852
9084172 500
9084668 496
9085168 500
9085660 492
9086160 500
9086656 496
9087156 500
9087648 492
9088152 504
9088644 492
9089140 496
9089636 496
9090132 496
9090628 496
9091128 500
9091620 492
9092120 500
9092612 492
9093108 496
9093608 500
9094100 492
9094600 500
9095092 492
9095592 500
9096084 492
9096584 500
9097076 492
9097576 500
9098072 496
9098572 500
9099064 492
9099564 500
9100056 492
9100556 500
9101052 496
9101552 500
9102044 492
9102544 500
9103040 496
9103540 500
9104032 492
9104532 500
9105024 492
9105524 500
9106016 492
9106516 500
9107012 496
9107512 500
9108008 496
9108504 496
9109000 496
9109492 492
9109992 500
9110488 496
9113092 2604
9113592 500
9114100 508
9114600 500
9115104 504
9115604 500
9116112 508
9116612 500
9117116 504
9117616 500
9118124 508
9118624 500
9119128 504
9119632 504
9120136 504
9120636 500
9121140 504
9121640 500
9122144 504
9122644 500
9123152 508
9123652 500
9124156 504
9124656 500
9125164 508
9125660 496
9126168 508
9126668 500
9127172 504
9127672 500
9128180 508
9128680 500
9129184 504
9129684 500
9130188 504
9130688 500
9131196 508
9131696 500
9132200 504
9132700 500
9133208 508
9133708 500
9134216 508
9134716 500
9135220 504

[incher91]

Hi Bill, thank you very much for your support! I think you solved my problem. I modified your code a litte bit, because i need to log the data ongoing. This is my new code: Fat16 fastDataLogger_gitHubIssue_v02.txt Occasionaly the writing duration jumps over 50000. But only every ~1000 lines, which is okay for my use.

Can i calculate after how many lines the 512 Blocks are "full"?

[greiman]

Can i calculate after how many lines the 512 Blocks are "full"?

Since lines vary in length, you would need to keep the total bytes written.

print()/println() returns the number of bytes written so you can keep the total bytes written.

  uint32_t byteCount = 0;

  // ...
  byteCount += file.print(item);
  // ...

  // byte used in 512 byte sector
  sectorBytes = byteCount%512

[greiman]

An easier way would be to use fileSize().

file.fileSize() returns the total bytes in the file including the cache buffer.