Ardunio SD.h lib faster?

mgk commented 8 years ago

More of a mystery than a bug, reporting on the off chance it is of interet.

I'm able to run the OpenNext.ino example at SPI_HALF_SPEED but at SPI_FULL_SPEED I get

SD errorCode: 0X4,0X33

I understand there are a lot of possible causes for this, in particular long paths in the SPI circuit. The reason I'm reporting this is that I can connect the same circuit at full speed using the CardInfo.ino example and the Arduino SD.h lib.

I found my way to the beta from https://github.com/greiman/SdFat/issues/16

(Looks like among other things, the beta configures the SPI layer to use automatically, which is nice. :+1:)

Using https://github.com/greiman/SdFat with #define SD_SPI_CONFIGURATION 1 I'm able to run on the M0 Pro, but as with SdFat-beta, only at half speed.

I've run with each s/w config a dozen times and the results are consistent.

The mystery is how the standard Arduino library could be faster than SdFat.

Environment

Board: Arduino M0 Pro
SD card reader connected by hw SPI pins, no other peripherals
no breadboard, jumper wires about 15cm long
Arduino.org IDE 1.7.8

greiman commented 8 years ago

Arduino SD.h runs at half speed. Also SD.h is just a wrapper for a 2009 version of SdFat.

boolean SDClass::begin(uint8_t csPin) {
  /*

    Performs the initialisation required by the sdfatlib library.

    Return true if initialization succeeds, false otherwise.

   */
  return card.init(SPI_HALF_SPEED, csPin) &&
         volume.init(card) &&
         root.openRoot(volume);
}

mgk commented 8 years ago

Yes, but the CardInfo example uses the wrapped SdFat lib directly, yes?

When I change:

https://github.com/arduino-org/Arduino/blob/ide-org-1.6.1.x/libraries/SD/examples/CardInfo/CardInfo.ino#L55

to

if (!card.init(SPI_FULL_SPEED, chipSelect)) {

the CardInfo example sketch works (presumably at full speed).

So I guess the mystery is why the wrapped 2009 SdFat lib is able to run at full speed when the modern ones are not.

greiman commented 8 years ago

The new SdFat is able to run much faster than the 2009 version.

The problem is an SPI error. That suggests a hardware problem with the new SdFat-beta having a different access pattern to the SPI bus.

Strange since the M0 Pro uses slow programmed I/O. While the SPI clock may be fast there will be a lot of dead time between bytes.

I have been optimizing SdFat-beta for new faster systems with DMA like the Particle Photon.

On the Due at full speed I get over 4 MB/sec read/write. What do you get with the bench example?

File size 20 MB
Buffer size 32768 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4150.43,20699,7318,7887

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4326.51,9505,7035,7571

I don't have a M0 Pro so I can't connect my scope and see why SD status is getting garbled over the SPI bus.

You might try enabling CRC to check for transmission errors. Edit SdFatConfig.h and set this

#define USE_SD_CRC 2

See if this catches more errors.

mgk commented 8 years ago

Ouch, yeah I'm only getting 140 KB/sec.

#define USE_SD_CRC 2 did not print additional errors.

Card Info

Card is class 10 SanDisk Extreme.

Type is FAT32 Card size: 15.93 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3 OEM ID: SD Product: SU16G Version: 8.0 Serial number: ... Manufacturing date: 4/2013

5 MB file / 512 B buffer tests File size 5 MB Buffer size 512 bytes Starting write test, please wait.

write speed and latency speed,max,min,avg KB/Sec,usec,usec,usec 140.61,30692,3182,3637 142.36,37502,3131,3592 141.97,55195,3141,3602 142.41,43009,3134,3591 142.15,60967,3139,3598 142.54,30061,3138,3588 139.97,85977,3103,3654 139.80,85114,3176,3658 140.14,83855,3179,3649 139.78,142024,3138,3659

Starting read test, please wait.

read speed and latency speed,max,min,avg KB/Sec,usec,usec,usec 237.66,4292,2052,2151 237.68,4289,2055,2151 237.70,4291,2055,2151 237.70,4289,2051,2151 237.68,4288,2052,2151

20 MB file / 16KB buffer tests

File size 20 MB Buffer size 16384 bytes

write speed and latency speed,max,min,avg KB/Sec,usec,usec,usec 285.16,77936,56682,57445 284.94,131935,56676,57491 284.81,129107,56679,57515

speed,max,min,avg KB/Sec,usec,usec,usec 297.05,57289,55043,55153 297.05,57290,55043,55153 297.05,57287,55042,55153

greiman commented 8 years ago

Wow that's less than a Uno.

Here is an Uno with a 32GB SanDisk Extreme.

Type is FAT32
Card size: 31.91 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
OEM ID: SD
Product: SE32G
Version: 8.0
Serial number: 0X838DE929
Manufacturing date: 9/2015

File size 5 MB
Buffer size 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
319.06,75704,1368,1596

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
501.72,1996,988,1014

Too bad the M0 Pro has a such a slow SPI driver

I am using SdFat on many other systems.

I bought a $19 Particle Photon:

https://store.particle.io/

I has 30 MHz SPI and I get over 3 MB/sec read/write with the stock SPI driver. It is a sweet WiFi development board.

I get over 7 MB/sec with this STM32 board using SDIO instead of SPI.

https://developer.mbed.org/platforms/Seeed-Arch-Max/

mgk commented 8 years ago

Thanks for all the background (and for SdFat itself!). Yeah the photon looks great. I'm using the M0 Pro as a dev board for a project that will embed a SAMD21.

Just fyi when I load the Arduino Zero bootloader and run with the arduino.cc libs it's even slower (100KB/s vs 140KB/s). Others have run into this too: https://forum.arduino.cc/index.php?topic=358160.30 using the patch from that thread and SdFat-beta I get 130KB/s with the arduino.cc libs.

For our project it will have to do, until a DMA SPI driver is available for the SAMD duino's.

comiati commented 7 years ago

Hi, Can you share the library for SD with bugs fixed? I use Arduino M0 with SD.h (standard library) and I have many problems, working just with some type of SD cards......and sometimes even the righit SD don't working (same SD sometime working perfect) Best regards

greiman commented 7 years ago

I am not planning on doing a custom driver for M0.

Also I don't support the old SD.h library that I wrote 2009. SD.h is maintained by the Arduino company.

greiman / SdFat-beta

Ardunio SD.h lib faster? #2

Environment

Card Info