Slow speeds writing to exFAT cards with RP2040 boards

[macster110]

I have been using your excellent SdFat library to write data to microSD cards but have encountered an issue with slow write speeds on RP2040 boards. I have tried using multiple SD cards (Fat32 and exFAT formatted), an Arduino Nano Connect RP2040 and Sparkfun Things Plus RP2040 board and I can only ever achieve write speeds of around 140kB/S dumping binary data into a file. I also tried a 32GB card (which had 140Kb/sec write speed with RP2040 boards) with an Arduino Uno and managed to get write speeds of around 700kB/sec.

It does not make much sense to me that an old Arduino Uno can write to an SD card faster than an RP2040 board? Does anyone have an ideas how I could speed up write speeds or what I might be doing wrong here?

The code I used to test is adapted from the SdFat library - see below. I have also tried writing blocks sequentially but that did from your ADC logger examples but that did not speed things up.

Hardware wise, the Sparkfun Things Plus has a built in SD card slot and I used this breakout board for the Arduino Nano Connect RP2040 and Arduino Uno.

Any help much appreciated.

/*
 * This program is a simple binary write/read benchmark.
 */
#include "SdFat.h"
#include "sdios.h"
#include "FreeStack.h"

// SD_FAT_TYPE = 0 for SdFat/File as defined in SdFatConfig.h,
// 1 for FAT16/FAT32, 2 for exFAT, 3 for FAT16/FAT32 and exFAT.
#define SD_FAT_TYPE 0
/*
  Change the value of SD_CS_PIN if you are using SPI and
  your hardware does not use the default value, SS.
  Common values are:
  Arduino Ethernet shield: pin 4
  Sparkfun SD shield: pin 8
  Adafruit SD shields and modules: pin 10
*/
// SDCARD_SS_PIN is defined for the built-in SD on some boards.
#ifndef SDCARD_SS_PIN
const uint8_t SD_CS_PIN = 9u; //9u for sparkfun things plus
#else  // SDCARD_SS_PIN
// Assume built-in SD is used.
const uint8_t SD_CS_PIN = SDCARD_SS_PIN;
#endif  // SDCARD_SS_PIN

// Try max SPI clock for an SD. Reduce SPI_CLOCK if errors occur.
#define SPI_CLOCK SD_SCK_MHZ(25)

// Try to select the best SD card configuration.
// #if HAS_SDIO_CLASS
// #define SD_CONFIG SdioConfig(FIFO_SDIO)
// #elif ENABLE_DEDICATED_SPI
#define SD_CONFIG SdSpiConfig(SD_CS_PIN, DEDICATED_SPI, SPI_CLOCK)
// #else  // HAS_SDIO_CLASS
// #define SD_CONFIG SdSpiConfig(SD_CS_PIN, SHARED_SPI, SPI_CLOCK)
// #endif  // HAS_SDIO_CLASS

// Set PRE_ALLOCATE true to pre-allocate file clusters.
const bool PRE_ALLOCATE = true;

// Set SKIP_FIRST_LATENCY true if the first read/write to the SD can
// be avoid by writing a file header or reading the first record.
const bool SKIP_FIRST_LATENCY = true;

// Size of read/write.
const size_t BUF_SIZE = 512;

// File size in MB where MB = 1,000,000 bytes.
const uint32_t FILE_SIZE_MB = 1;

// Write pass count.
const uint8_t WRITE_COUNT = 2;

// Read pass count.
const uint8_t READ_COUNT = 2;
//==============================================================================
// End of configuration constants.
//------------------------------------------------------------------------------
// File size in bytes.
const uint32_t FILE_SIZE = 1000000UL*FILE_SIZE_MB;

// Insure 4-byte alignment.
uint32_t buf32[(BUF_SIZE + 3)/4];
uint8_t* buf = (uint8_t*)buf32;

#if SD_FAT_TYPE == 0
SdFat sd;
File file;
#elif SD_FAT_TYPE == 1
SdFat32 sd;
File32 file;
#elif SD_FAT_TYPE == 2
SdExFat sd;
ExFile file;
#elif SD_FAT_TYPE == 3
SdFs sd;
FsFile file;
#else  // SD_FAT_TYPE
#error Invalid SD_FAT_TYPE
#endif  // SD_FAT_TYPE

// Serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
// Store error strings in flash to save RAM.
#define error(s) sd.errorHalt(&Serial, F(s))
//------------------------------------------------------------------------------
void cidDmp() {
  cid_t cid;
  if (!sd.card()->readCID(&cid)) {

    error("readCID failed");
  }
  cout << F("\nManufacturer ID: ");
  cout << hex << int(cid.mid) << dec << endl;
  cout << F("OEM ID: ") << cid.oid[0] << cid.oid[1] << endl;
  cout << F("Product: ");
  for (uint8_t i = 0; i < 5; i++) {
    cout << cid.pnm[i];
  }
  cout << F("\nVersion: ");
  cout << int(cid.prv_n) << '.' << int(cid.prv_m) << endl;
  cout << F("Serial number: ") << hex << cid.psn << dec << endl;
  cout << F("Manufacturing date: ");
  cout << int(cid.mdt_month) << '/';
  cout << (2000 + cid.mdt_year_low + 10 * cid.mdt_year_high) << endl;
  cout << endl;
}
//------------------------------------------------------------------------------
void clearSerialInput() {
  uint32_t m = micros();
  do {
    if (Serial.read() >= 0) {
      m = micros();
    }
  } while (micros() - m < 10000);
}
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
  //pinMode(SD_CS_PIN, OUTPUT);
  // Wait for USB Serial
  while (!Serial) {
    yield();
  }
  delay(1000);
  cout << F("\nUse a freshly formatted SD for best performance.\n");
  Serial.println(SD_CS_PIN);

     pinMode(SD_CS_PIN, OUTPUT);
  if (!ENABLE_DEDICATED_SPI) {
    cout << F(
      "\nSet ENABLE_DEDICATED_SPI nonzero in\n"
      "SdFatConfig.h for best SPI performance.\n");
  }
  // use uppercase in hex and use 0X base prefix
  cout << uppercase << showbase << endl;
}
//------------------------------------------------------------------------------
void loop() {
  float s;
  uint32_t t;
  uint32_t maxLatency;
  uint32_t minLatency;
  uint32_t totalLatency;
  bool skipLatency;

  // Discard any input.
  clearSerialInput();

  // // F() stores strings in flash to save RAM
  // cout << F("Type any character to start\n");
  // while (!Serial.available()) {
  //   yield();
  // }
#if HAS_UNUSED_STACK
  cout << F("FreeStack: ") << FreeStack() << endl;
#endif  // HAS_UNUSED_STACK

  if (!sd.begin(SD_CONFIG)) {
    sd.initErrorHalt(&Serial);
  }
  if (sd.fatType() == FAT_TYPE_EXFAT) {
    cout << F("Type is exFAT") << endl;
  } else {
    cout << F("Type is FAT") << int(sd.fatType()) << endl;
  }

  cout << F("Card size: ") << sd.card()->sectorCount()*512E-9;
  cout << F(" GB (GB = 1E9 bytes)") << endl;

  cidDmp();

  // open or create file - truncate existing file.
  if (!file.open("bench.dat", O_RDWR | O_CREAT | O_TRUNC)) {
    error("open failed");
  }

  // fill buf with known data
  if (BUF_SIZE > 1) {
    for (size_t i = 0; i < (BUF_SIZE - 2); i++) {
      buf[i] = 'A' + (i % 26);
    }
    buf[BUF_SIZE-2] = '\r';
  }
  buf[BUF_SIZE-1] = '\n';

  cout << F("FILE_SIZE_MB = ") << FILE_SIZE_MB << endl;
  cout << F("BUF_SIZE = ") << BUF_SIZE << F(" bytes\n");
  cout << F("Starting write test, please wait.") << endl << endl;

  // do write test
  uint32_t n = FILE_SIZE/BUF_SIZE;
  cout <<F("write speed and latency") << endl;
  cout << F("speed,max,min,avg") << endl;
  cout << F("KB/Sec,usec,usec,usec") << endl;
  for (uint8_t nTest = 0; nTest < WRITE_COUNT; nTest++) {
    file.truncate(0);
    if (PRE_ALLOCATE) {
      if (!file.preAllocate(FILE_SIZE)) {
        error("preAllocate failed");
      }
    }
    maxLatency = 0;
    minLatency = 9999999;
    totalLatency = 0;
    skipLatency = SKIP_FIRST_LATENCY;
    t = millis();
    for (uint32_t i = 0; i < n; i++) {
      uint32_t m = micros();
      if (file.write(buf, BUF_SIZE) != BUF_SIZE) {
        error("write failed");
      }
      m = micros() - m;
      totalLatency += m;
      if (skipLatency) {
        // Wait until first write to SD, not just a copy to the cache.
        skipLatency = file.curPosition() < 512;
      } else {
        if (maxLatency < m) {
          maxLatency = m;
        }
        if (minLatency > m) {
          minLatency = m;
        }
      }
    }
    file.sync();
    t = millis() - t;
    s = file.fileSize();
    cout << s/t <<',' << maxLatency << ',' << minLatency;
    cout << ',' << totalLatency/n << endl;
  }
  cout << endl << F("Starting read test, please wait.") << endl;
  cout << endl <<F("read speed and latency") << endl;
  cout << F("speed,max,min,avg") << endl;
  cout << F("KB/Sec,usec,usec,usec") << endl;

  file.close();
}

[greiman]

It does not make much sense to me that an old Arduino Uno can write to an SD card faster than an RP2040 board?

It's the mbed SPI driver used in the Arduino package. I bought a number of RP2040 boards and was disappointed.

I also found a basic flaw in the ADC, only good to about 8-9 bits, so I gave up using the RP2040.

You might try editing SdFatConfig.h and set USE_SPI_ARRAY_TRANSFER to one at about line 130. I have not tried this lately on RP2040.

/**
 * If USE_SPI_ARRAY_TRANSFER is non-zero and the standard SPI library is
 * use, the array transfer function, transfer(buf, size), will be used.
 * This option will allocate up to a 512 byte temporary buffer for send.
 * This may be faster for some boards.  Do not use this with AVR boards.
 */
#ifndef USE_SPI_ARRAY_TRANSFER
#define USE_SPI_ARRAY_TRANSFER 0
#endif  // USE_SPI_ARRAY_TRANSFER

[macster110]

Thanks Greiman, that is good to know. Could you elaborate a bit more on the ADC issue?

[macster110]

You might try editing SdFatConfig.h and set USE_SPI_ARRAY_TRANSFER to one at about line 130. I have not tried this lately on RP2040.

Wow, that worked! The SD card write speeds are now 1700 kB/sec - thanks very much! I was really happy with this library so very pleased I can use it still! Thanks for all the help.

I'd still be keen to hear about this ADC issue only being good to 8-9 bits? I know there is a well documented issue with some non linearlity - is this what you mean?

[greiman]

See sections "4.9.3. ADC ENOB" and "4.9.4. INL and DNL" on page 588 of this document:

https://datasheets.raspberrypi.com/rp2040/rp2040-datasheet.pdf

It is a fundamental design problem with the SAR ADC caps. ENOB typically 8-9.