esp32 spi_speed problem

[nices0325]

I was using Arduino Due with sdFat library. SPI_SPEED that I 'd set was 40mHz and, It worked well. Now then I'm using esp32 , but the maximum speed of SPI_SPEED seems to be 16mHz. I don't know why, If I set higher value over 16mHz, sd card module can't initialize. What 'd be the problem.. I searched that esp32 spi speed goes up to 80mHz and , sdFat library support for 50mHz speed of sdcard module. Here's my code

define SPI_SPEED SD_SCK_MHZ(24)

void setup() { if (!sd.begin(chipSelect, SPI_SPEED)) { //sd.initErrorHalt(); Serial.println("Card failed, or not present");

    digitalWrite(green, HIGH);
    digitalWrite(red, LOW);
    digitalWrite(blue, HIGH);

    t.tone(BUZZER, beepFrequencyF, beepDurationF);

    //return;
}

else if (sd.begin(chipSelect, SPI_SPEED))
{
digitalWrite(green, LOW);
digitalWrite(red, HIGH);
digitalWrite(blue, HIGH);

t.tone(BUZZER, beepFrequencyS2, beepDurationS);
delay(250);
Serial.println("card initialized.");

}

[greiman]

SD cards require full duplex SPI. The card looks at MOSI while it write to MISO.

ESP quote:

While in general, speeds up to 80MHz on the dedicated SPI pins and 40MHz on GPIO-matrix-routed pins are supported, full-duplex transfers routed over the GPIO matrix only support speeds up to 26MHz.

I believe the driver uses the GPIO-martix so ESP32 is flaky anywhere near 26 MHz.

I have given up on anything above 16 MHz and even that will be slow because of driver overhead.

Due goes fast since I wrote a custom driver when it appeared. Now I depend on vendors to write SPI drivers since SdFat is used on about a 100 boards.

[nices0325]

Thanks for your reply!! I appreciate that. Would it be more faster to use this library with esp32, using lower times of sdFat.begin() function. In my code, I use this function quite a lot, due to checking sd card module work properly. Is there any function to check sdcard module returns true when work properly?

[greiman]

Probably best to use the ESP SD library. I don't want to waste time trying to support SdFat on ESP32, there are too many conflicts.

[nices0325]

아마도 ESP SD 라이브러리를 사용하는 것이 가장 좋습니다. ESP32에서 SdFat을 지원하는 데 시간을 낭비하고 싶지 않습니다. 충돌이 너무 많습니다.

Thanks for you reply!!

[JimDrewGH]

The ESP32 in 4 bit (MMC) mode can read >17MB/s and write nearly 10MB/s. You just have to put the driver into HS (high speed) mode. I am not sure why this is not more widely known as it was resolved more than 4 years ago!

https://esp32.com/viewtopic.php?t=2055

[squonk11]

The ESP32 in 4 bit (MMC) mode can read >17MB/s and write nearly 10MB/s. You just have to put the driver into HS (high speed) mode. I am not sure why this is not more widely known as it was resolved more than 4 years ago!

https://esp32.com/viewtopic.php?t=2055

yes, but if I read the thread you are citing correctly, then the high read speed is achieved only in blockread mode; in normal fread() it is much slower (1MB/s)?

[greiman]

Unfortunately the 4-bit driver would need to be rewritten to get high speed in most programs using smaller reads and writes.

I rewrote the driver for Teensy 3.x and 4.x and get 22 MB/s for 512 byte reads and writes. The same is true for many other boards. I can't rewrite and maintain drivers for all boards that could benefit.

You can provide your own external driver. Here is an example wrapper that works for USB keys.

#ifndef UsbMscDriver_h
#define UsbMscDriver_h
#include "SdFat.h"

// If the next include fails, install the USB_HOST_SHIELD library.
#include <masstorage.h>
//------------------------------------------------------------------------------
/** Simple USB init function.
 * \param[in] usb USB host object to be initialized.
 * \return true for success or false for failure.
 */
bool initUSB(USB* usb);
//------------------------------------------------------------------------------
/** Print debug messages if USB_FAT_DBG_MODE is nonzero */
#define USB_FAT_DBG_MODE 1
//------------------------------------------------------------------------------
/** Maximum time to initialize the USB bus */
#define TIMEOUT_MILLIS 400000L
class UsbMscDriver : public BlockDeviceInterface {
 public:
  UsbMscDriver(BulkOnly* bulk)  : m_lun(0), m_bulk(bulk) {}
  bool isBusy() {return false;}
  bool readSector(uint32_t sector, uint8_t* dst) {
    uint8_t rc = m_bulk->Read(m_lun, sector, 512, 1, dst);
    return 0 == rc;
  }
  uint32_t sectorCount() {
    return m_bulk->GetCapacity(m_lun);
  }
  bool syncDevice() {return true;}
  bool writeSector(uint32_t sector, const uint8_t* src) {
    return  0 == m_bulk->Write(m_lun, sector, 512, 1, src);    
  }

  bool readSectors(uint32_t sector, uint8_t* dst, size_t ns) {
    return 0 == m_bulk->Read(m_lun, sector, 512, ns, dst);   
  }
  bool writeSectors(uint32_t sector, const uint8_t* src, size_t ns) {
    return  0 == m_bulk->Write(m_lun, sector, 512, ns, src);      
  }

 private:
  uint8_t m_lun;
  BulkOnly* m_bulk;   
};
#endif  // UsbMscDriver_h

Here is an example program that uses the above wrapper for the USB mass storage driver.

// Edit SdFatConfig.h and enable generic block devices.
// #define USE_BLOCK_DEVICE_INTERFACE 1
#include "UsbMscDriver.h"

USB usb;
BulkOnly bulk(&usb);
UsbMscDriver usbKey(&bulk);

FsVolume key;
FsFile file;

void setup() {
  Serial.begin(9600);
  while (!Serial) {}
  Serial.println(F("Type any character to start"));
  while (!Serial.available()) {}

  if (!initUSB(&usb)) {
    Serial.println("initUSB failed");
    while(1){}
  }
  // Must set USE_BLOCK_DEVICE_INTERFACE non-zero in SdFatConfig.h
  if (!key.begin(&usbKey)) {
    Serial.println("key.begin failed");
    while(1) {}
  }
  if (!file.open("usbtest.txt", FILE_WRITE)) {
    Serial.println("file.open failed");
    while(1) {}    
  }
  file.println("test line");
  file.close();
  key.ls(LS_DATE | LS_SIZE);
}
void loop() {
}

[squonk11]

sorry for not being clear enough: I was referring to the post of JimDrewGH who stated that the esp-idf fat driver can be accelerated to 17MB/sec by using a special option in esp-idf. But this speed (17MB/sec read) can only be reached in some king of blockread and not via the stadard fread() function.

[greiman]

I was talking about the sdmmc driver in esp-idf. It is extremely slow for small transfers, For writes of 512 bytes, only a few KB/sec. Print and small binary writes are way slower than SPI.

ESP32 is not the only system with this problem, STM32 has the problem in it's Arduino board package.

SD cards require huge transfers for full speed writes. In my SDIO driver on Teensy I keep the SD in write mode as long as possible up to gigabytes.

The EPS32 ends write mode after every transfer so you must do you must do 64KB block writes to get 10 MB/sec.

Systems like IOS, Android, Linux, MacOS, and Windows use huge disk caches to increase performance. These systems can cache a GB or more and do 100 MB transfers.

[greiman]

I tried a medium quality Samsung 64GB UHS-I microSD with CrystalDiskMark on Windows. Result is read 93 MB/sec write 88 MB/sec. About the same for SanDisk UHS-I card, read 95 MB/sec write 86 MB/sec.

UHS-II cards with 8-bit interface are about twice as fast. Big caches, 1.8V signaling and 8-bit bus give about 200 MB/sec on real OSes.

[squonk11]

o.k. now I think I got it. But is it possible to use SDFat with esp-idf (not Adruino) in order to speed up SD card reads? Does SDFat work with ESP32 out of the box or does it need some tweeks/adaptions?

[greiman]

Read will also be slow. Most transfers are one sector and at best esp-idf will probably be under 1 MB/sec. I get about 2 MB/sec using SPI on the new $4.00 Pi Pico RP2040 with dedicated SPI on its second SPI port for one sector transfers.

As I said above, you can try esp-idf by using an external wrapper for the esp-idf driver. If you do large reads you may get high speeds.

The esp MMCSD controller has lots of problems. With file I/O you will need to make all transfers a multiple of four bytes or high overhead will occur because of memory alignment problems with DMA.

Just replace USB calls in class UsbMscDriver : public BlockDeviceInterface { with esp-idf calls in the above USB driver wrapper.

Then look at the above program for use of your esp-idf wrapper.

[Miguel0101]

@greiman, is there a way to connect a pen drive to an ESP32 using UsbMscDriver?

[greiman]

@Miguel0101, I don't know if ESP32 has kernel support for USB MSC. If it does you can modify the above example to wrap the ESP32 API.

I can't help since I don't have time or interest in supporting USB flash drives on ESP32.