Add support for the Nano 33 IoT

[ArjanPalsje]

My code which was working fine on version 1.1.4. Doesn't work anymore by using the newest version of the library (version 2.0.2). It does compile, but no file is created on the SD card. When I run an example sketch it is not detecting the SD card. But with the same setup, version 1.1.4 does work. Plz fix this

[greiman]

The Nano 33 IoT has a SAMD21 processor. I tested SAMD21 by running the QuickStart example on a MRKZERO board and it works.

I don't have a Nano IoT so I need more info.

Have you tried the QuickStart example?

Which example did you run?

Did you edit the example?

What was the output?

What kind of SD module are you using?

Which pins are connected to the module?

[tickietackie]

Actually I am facing similar issues, also with an Arduino Nano 33 IOT. But I suspect it has nothing to do with the Chip itself. Below is I think the important parts of the code. Quickstart is working fine. So I am quite sure everything is connected correctly. Pins used: 13:CLK, 12:MISO,11:MOSI,10 CS, 5V port of Arduino IOT and one of the ground pins. Delays are just to be sure it has nothing to do with timing stuff.

SD Card: Intenso Micro SDHC 8GB SD Shield: AZDelivery 3 x Set SPI Reader Micro Speicher SD TF Karte Memory Card Shield

//in header
#define CHIP_SELECT 10
// SD_FAT_TYPE = 0 for SdFat/File as defined in SdFatConfig.h,
// 1 for FAT16/FAT32, 2 for exFAT, 3 for FAT16/FAT32 and exFAT.
#define SD_FAT_TYPE 1
#define DISABLE_CHIP_SELECT -1
#define SPI_SPEED SD_SCK_MHZ(4)   //Had to reduce the speed to 4 because it would not work above

/* --------------- HeaderFunctions ---------------------------- */

bool initializeSdCard();
bool loadConfiguration(const char *filename, TableConfig &config);
bool saveConfiguration(const char *filename, const TableConfig &config);
void printFileToSerial(const char *filename);

//in cpp
bool initializeSdCard() {
  // Initialize the SD.
  if (!SD.begin(CHIP_SELECT, SPI_SPEED)) {
    SD.initErrorPrint(&Serial);
    Serial.println("Failed to initialize SD card");
    return false;
  } else {
    return true;
    Serial.println("SD card successfully initialized.");
  }
}

// Loads the configuration from a file
bool loadConfiguration(const char *filename, TableConfig &config) {
  // Open file for reading
  delay(1000);
  File file = SD.open(filename);
  if (!file) {
    file.close();
    Serial.println("Cant load the file");
    return false;
  }

  // Parse the JSON object in the file
  bool success = deserializeConfig(file, config);

  // This may fail if the JSON is invalid
  if (!success) {
    file.close();
    Serial.println(F("Failed to deserialize configuration"));
    return false;
  }

  // Close the file (Curiously, File's destructor doesn't close the file)
  file.close();
  return true;
}

// Saves the configuration to a file
bool saveConfiguration(const char *filename, const TableConfig &config) {
  // Delete existing file, otherwise the configuration is appended to the file
  bool success;
  Serial.println(filename);
  delay(1000);
  if (SD.exists(filename)) {
    success = SD.remove(filename);
    if (!success) {
      Serial.println("Deleting file failed.");
      return false;
    }
  } else {
    Serial.println("No file on SD-Card. No deletion necessary.");
  }

  delay(1000);
  // Open file for writing
  Serial.println(filename);
  File fileToWrite = SD.open(filename, FILE_WRITE);
  if (!fileToWrite) {
    fileToWrite.close();
    Serial.println(F("Failed to open config file for write"));
    return false;
  }

  // Serialize JSON to file
  success = serializeConfig(config, fileToWrite);
  if (!success) {
    fileToWrite.close();
    Serial.println(F("Failed to serialize configuration"));
    return false;
  }

  // Close the file
  fileToWrite.close();
  return true;
}

// Prints the content of a file to the Serial
void printFileToSerial(const char *filename) {
  // Open file for reading
  File fileToRead = SD.open(filename);
  if (!fileToRead) {
    fileToRead.close();
    Serial.println(F("Failed to read file"));
    return;
  }

  // Extract each characters by one by one
  for (int i = 0; i < 200; i++) {
    Serial.print((char)fileToRead.read());
  }
  Serial.println();

  // Close the file
  fileToRead.close();
}

This is I think the important part of the code ...

So the problem is, the following: The first time I run the code a file is created. I can see it when I plug the SD card into my computer. The json is save fine. But the second the time the file is not recognized correctly. So it cannot be opened in any way.

Program flow:

1. open file if its there and read content
2. delete file if its there
3. write content again.

So the output is:

Cant load the file
Using default config
/config.json
No file on SD-Card. No deletion necessary.
/config.json
Failed to open config file for write
Failed to read file
Restart to load the config

How to debug this further?

[greiman]

I now have an Nano 33 IoT. I have tested many of the examples and it works with my SD socket and SD cards.

I use a simple microSD socket with no level shifter since level shifters are not needed for 3.3V chips. Sometimes level shifters distort signals for 3.3V chips like SAMD.

I tend to use SanDisk or Samsung SDs since some other brands work fine for the 4-bit SDIO bus used in phones, PC/Mac and other devices but have problems with the SPI protocol. You could try another SD card to see if behavior changes.

To debug I suggest you check for an SPI/hardware error whenever a call fails like this:

if (SD.sdErrorCode()) { SD.errorPrint("message about error"); }

If there is no SPI/hardware error you can enable debug messages and find where the error occurs.

Edit SdFat/src/common/DebugMacros.h at about line 28 and set USE_DBG_MACROS non-zero.

[tickietackie]

Ok, I bought a Sandisk ... Same behavior.

First question: When should I use SD_FAT_TYPE 0 and when SD_FAT_TYPE 1. My trial and error tells me ArduinoJson only is satisfied with the normal File, so Type 0. But maybe this was just a false conclusion. I read the documentation, but I am not really getting it. Can this have sth. to do with the issues? I tried both, but I am not succeeding so far.

I also tried the debug mode and thats the output:

DBG_FAIL: ExFatPartition.cpp.302
SD card successfully initialized.
DBG_FAIL: FatFileLFN.cpp.423
Couldn't open the file to read
Using default config
DBG_FAIL: FatFileLFN.cpp.423
Not necesssary to delete file
DBG_FAIL: FatFileLFN.cpp.423
Failed to open config file for write
DBG_FAIL: FatFileLFN.cpp.423
Couldn't open the file to read
Restart to load the config

So it looks like it immediately fails trying to open the volume. I pretty sure you might have another clue for me now? At least I hope so. I tried to debug it further in the library code, but didn't get that far.

PS: Seems like its failing with something in ExFat during init? Which is strange because my SD Card is formatted as FAT32.

[greiman]

The failure with exFAT is a check for an exFAT volume. You are using FAT and that succeeds. If you only want FAT32/FAT16 support, edit SdFatConfig.h at about line 81 and set SDFAT_FILE_TYPE to 1 for ARM boards.

Your open for write fails here in SdFat:

 create:
  // don't create unless O_CREAT and write mode
  if (!(oflag & O_CREAT) || !isWriteMode(oflag)) {
    DBG_FAIL_MACRO;
    goto fail;
  }

This seems impossible since you use open(filename, FILE_WRITE).

Try running this sketch. It works on my Nano 33 IOT.

#include "SdFat.h"
#define CHIP_SELECT 10
#define SPI_SPEED SD_SCK_MHZ(4)
#define TEST_FILENAME "IOT33.txt"
SdFat SD;

void openTest() {
  File file = SD.open(TEST_FILENAME, FILE_WRITE);
  if (!file) {
    Serial.println("file open failed");
  } else {
    Serial.println("file opened");
    if (!file.remove()) {
      Serial.println("remove failed");
    } else {
      Serial.println("file removed");
    }
  }
}

void setup() {
  Serial.begin(9600);
  while (!Serial) {}
  Serial.println("Type any character to test open");
  while (!Serial.available()) {}
  Serial.println("start test");
  if (!SD.begin(CHIP_SELECT, SPI_SPEED)) {
    Serial.println("SD.begin failed");
  }
  openTest();
}
void loop() {}

Here is the output:

Type any character to test open
start test
file opened
file removed

If it works on your Nano 33 IOT then something you are using in your sketch must conflict with the definition of FILE_WRITE.

[tickietackie]

OK, yes, this seems to work. Thanks for the support and quick help so far. I will try to build the application use case the basic code example and let you know which part interfered here.

[tickietackie]

load TablePartConfiguration
load Effect
load TablePartConfiguration
load Effect
DBG_FAIL: FsCache.cpp.41
DBG_FAIL: FatFile.cpp.795
DBG_FAIL: FatFile.cpp.870
DBG_FAIL: FatFileLFN.cpp.332
Failed to create config file
DBG_FAIL: FsCache.cpp.41
DBG_FAIL: FatFile.cpp.795
DBG_FAIL: FatFile.cpp.870
DBG_FAIL: FatFileLFN.cpp.332

Would this indicate some cache overflow, or Ram overflow? Maybe my file I am trying to load is to big?

also can happen when printing the file:

{
  "name": "test",
  "id": 0,
  "status": 1,
  "configurationPerState": {
    "1": {
      "1": {
        "maximumBrightness": 0
      },
      "2": {
        "maximumBrightness": 0
      },
      "3": {
        "maximumBrightness": 0
      },
      "4": {
        "maximumBrightness": 0
      }
    },
    "2": {
      "1": {
        "maximumBrightness": 0
      },
      "2": {
        "maximumBrightness": 0
      },
      "3": {
        "maximumBrightness": 0
      },
      "4"DBG_FAIL: FsCache.cpp.41
DBG_FAIL: FatFile.cpp.795
⸮DBG_FAIL: FsCache.cpp.41
DBG_FAIL: FatFile.cpp.795
⸮DBG_FAIL: FsCache.cpp.41
.... //never ends, is stuck here

[tickietackie]

Further investigating revealed that this part is letting it fail: Removing the print will let it run through without problems. Looks like this isn't really efficient then 😅

void printFile(const char *filename) {
  // Open file for reading
  File file = SD.open(filename, FILE_READ);
  if (!file) {
    //File file;
    //if (!file.open(filename, FILE_READ)) {
    Serial.println("Couldn't open the file to read");
    return;
  }
  else {
    Serial.println("Successfully opened the file for reading");
  }

  // Extract each characters by one by one
  for (int i = 0; i < 200; i++) {
    Serial.print((char)file.read());
  }
  Serial.println();

  // Close the file
  file.close();
}

[tickietackie]

Ok ..... forget what I was saying. It is working now. I had to reduce the speed even more to 1MHz. Now it is working. Seems like it was somewhat on the limit and therefore it was crashing for bigger files. Thanks for helping and maybe this is somewhat useful for others, so this is my working code to read and write a json from / to an sd card:

#include "SdFat.h"
#define CHIP_SELECT 10
#define SPI_SPEED SD_SCK_MHZ(1)
#define TEST_FILENAME "config.json"
#include <ArduinoJson.h>

SdFat SD;

struct Battery {
  int  cycleCount;
  byte level;
  byte health;
  int maxVoltage;
  int designCapacity;
  int fullChargeCapacity;

  void load(JsonObjectConst);
  void save(JsonObject) const;
};

struct TableConfig {
  char name[20];
  int id;
  int status;
  Battery battery;

  void load(JsonObjectConst);
  void save(JsonObject) const;
};

TableConfig config;

bool serializeConfig(const TableConfig &config, Print &dst);
bool deserializeConfig(Stream &src, TableConfig &config);

void Battery::save(JsonObject obj) const {
  obj["cycle_count"] = cycleCount;
  obj["level"] = level;
  obj["health"] = health;
  obj["max_voltage"] = maxVoltage;
  obj["design_capacity"] = designCapacity;
  obj["full_charge_capacity"] = fullChargeCapacity;
}

void Battery::load(JsonObjectConst obj) {
  Serial.println(F("load Battery"));
  cycleCount = obj["cycle_count"] | 0;
  level = obj["level"] | 0;
  health = obj["health"] | 0;
  maxVoltage = obj["max_voltage"] | 0;
  designCapacity = obj["design_capacity"] | 0;
  fullChargeCapacity = obj["full_charge_capacity"] | 0;
}

void TableConfig::load(JsonObjectConst obj) {
  Serial.println(F("load TableConfig"));
  strlcpy(name, obj["name"] | "", sizeof(name));
  id = obj["id"] | 0;
  status = obj["status"] | 0;
  battery.load(obj["battery"]);

}

void TableConfig::save(JsonObject obj) const {
  Serial.println(F("save TableConfig"));
  obj["name"] = name;
  obj["id"] = id;
  obj["status"] = status;

  battery.save(obj.createNestedObject("battery"));
}

bool serializeConfig(const TableConfig &config, Print &dst) {
  DynamicJsonDocument doc(2084);

  // Create an object at the root
  JsonObject root = doc.to<JsonObject>();
  Serial.println(F("Start serialization"));
  // Fill the object
  config.save(root);

  // Serialize JSON to file
  return serializeJsonPretty(doc, dst) > 0;
}

bool deserializeConfig(Stream &src, TableConfig &config) {
  DynamicJsonDocument doc(2084);

  // Parse the JSON object in the file
  DeserializationError err = deserializeJson(doc, src);
  if (err) {
    Serial.print(F("deserializeJson() failed: "));
    Serial.println(err.c_str());
  }

  config.load(doc.as<JsonObject>());
  return true;
}

// Loads the configuration from a file on SD
bool loadFile(const char *filename, TableConfig &config) {
  // Open file for reading
  File file = SD.open(TEST_FILENAME, FILE_READ);

  // This may fail if the file is missing
  if (!file) {
    Serial.println(F("Failed to open config file"));
    return false;
  }

  // Parse the JSON object in the file
  bool success = deserializeConfig(file, config);

  // This may fail if the JSON is invalid
  if (!success) {
    Serial.println(F("Failed to deserialize configuration"));
    file.close();
    return false;
  }
  file.close();
  return true;
}

// Saves the configuration to a file on SD
void saveFile(const char *filename, const TableConfig &config) {
  if (SD.exists(filename)) {
    Serial.println("Removing file before writing");
    SD.remove(filename);
  } else {
    Serial.println("Not necesssary to delete file");
  }

  File file = SD.open(TEST_FILENAME, FILE_WRITE);
  if (!file) {
    Serial.println(F("Failed to create config file"));
    return;
  }

  // Serialize JSON to file
  bool success = serializeConfig(config, file);
  if (!success) {
    Serial.println(F("Failed to serialize configuration"));
  }

  file.close();
}

// Prints the content of a file to the Serial
void printFile(const char *filename) {
  // Open file for reading
  File file = SD.open(TEST_FILENAME, FILE_READ);
  if (!file) {
    Serial.println(F("Failed to open config file"));
    return;
  }

  // Extract each by one by one
  while (file.available()) {
    Serial.print((char)file.read());
  }
  file.close();
  Serial.println();
}

void setup() {
  // Initialize serial port

  Serial.begin(9600);
  while (!Serial) {}
  Serial.println(F("Type any character to test open"));
  while (!Serial.available()) {}
  if (!SD.begin(CHIP_SELECT, SPI_SPEED)) {
    SD.initErrorPrint(&Serial);
    Serial.println("Failed to initialize sd card");
    return ;
  }

  // Load configuration
  bool loaded = loadFile(TEST_FILENAME, config);
  //bool loaded = false;
  if (!loaded) {
    Serial.println(F("Using default config"));

    strcpy(config.name, "test");
    config.id = 0;
    config.status = 1;
    config.battery.cycleCount = 0;
    config.battery.level = 0;
    config.battery.health = 100;
    config.battery.maxVoltage = 0;
    config.battery.designCapacity = 1000;
    config.battery.fullChargeCapacity = 1000;
  }
  //else {
  //Serial.println(F("Loaded file successfully"));
  //}

  // Save configuration

  saveFile(TEST_FILENAME, config);

  // Dump config file
  printFile(TEST_FILENAME);

  if (!loaded)
    Serial.println(F("Restart to load the config"));
  else
    Serial.println(F("Done!"));
}

void loop() {
}

[greiman]

Would this indicate some cache overflow, or Ram overflow? Maybe my file I am trying to load is to big?

The cache will not overflow. It is just a 512 byte buffer for an SD sector.

I test high end boards with huge files, 4GB FAT and 10GB for exFAT on Teensy 4.1 boards. Teensy SDIO does read/write at 22 MB/sec so 10GB only takes 455 seconds.

The cache error is an SD read error. Try the SdFat bench example to see if your card has problems.

If I set the SPI rate to 8 MHz: #define SPI_CLOCK SD_SCK_MHZ(8)

The bench write/read rates for an 5 MB file are:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
408.16,1276,1248,1251
408.10,1274,1248,1252

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
419.67,1225,1214,1217
419.67,1225,1214,1217

My IOT will run at 12 MHz but read/write rates are about the same.

The SAMD SPI driver limits clock to 12MHz.

#if defined(ARDUINO_ARCH_SAMD)
  // The datasheet specifies a typical SPI SCK period (tSCK) of 42 ns,
  // see "Table 36-48. SPI Timing Characteristics and Requirements",
  // which translates into a maximum SPI clock of 23.8 MHz.
  // Conservatively, the divider is set for a 12 MHz maximum SPI clock.
  #define SPI_MIN_CLOCK_DIVIDER (uint8_t)(1 + ((F_CPU - 1) / 12000000))
#endif

[tickietackie]

That's what I am getting with 8MHz:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
418.41,9540,1211,1221
419.18,9644,1211,1219

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
403.71,1273,1261,1265
403.75,1273,1261,1265

Done

[greiman]

I have no idea why your app fails in various ways. Except for the Arduino SAMD SPI driver there is no SAMD dependent code in SdFat.

[tickietackie]

Would you be so kind to run the code I've posted above? https://github.com/greiman/SdFat/issues/228#issuecomment-834663225

It should run out of the box. You just need the ArduinoJson lib. If this is working for you for 1MHz and maybe 8MHz, it should be an hardware issue on my side. Really appreciate your will to help me!

[tickietackie]

Actually this not longer seems to be necessary. I changed all my wires, just to be sure and also I made them shorter .... aaaaaand now its working, even with 8MHz. I don't know why it didn't before. But I don't really care at that point, because that's not my major issue 😄. Thanks for your support! This is really not be expected for such a free library.

[tickietackie]

Still can't get it to work reliable: Reading works when a file is on the SD card, but as soon as I want to write it fails. Also SD.errorPrint or SD.sdErrorCode() lets the Arduino hang.

// Saves the configuration to a file
bool SdCardHandler::writeToFile(const char *filePath,
                                const TableConfig &config) {
  if (!SdCardHandler::initialized) {
    return false;
  }

  if (SD.exists(filePath)) {
    DEBUG_LN(F("Removing file before writing config"));
    SD.remove(filePath);
  } else {
    DEBUG_LN(F("Not necesssary to delete file"));
  }

  File file = SD.open(filePath, O_WRONLY);
  if (!file) {
    SD.errorPrint(&Serial);
    DEBUG_LN(F("Failed to open file to write config"));
    /*if (SD.sdErrorCode()) {
      SD.errorPrint("message about error");
    }*/
    return false;
  }

  // Serialize JSON to file
  bool success = config.serializeConfig(file);
  if (!success) {
    DEBUG_LN(F("Failed to serialize configuration"));
    file.close();
    return false;
  } else {
    DEBUG_LN(F("Successfully serialize configuration"));
  }

  // Close the file
  file.close();
  return true;
}

Output with Debug mode and trying to get the error message:

DBG_FAIL: ExFatPartition.cpp.302
SD card successfully initialized.
DBG_FAIL: FatFileLFN.cpp.423
File (config) not found
DBG_FAIL: FatFileLFN.cpp.423
File not found. Could not print.
Not necesssary to delete file
..... hanging

Output when not trying to get the error with SD.errorPrint(&Serial):

DBG_FAIL: ExFatPartition.cpp.302 --> we already clarified that above
SD card successfully initialized.
DBG_FAIL: FatFileLFN.cpp.423. --> This makes sense right?, because I want to read here not write
File (config) not found
DBG_FAIL: FatFileLFN.cpp.423.  --> Same
File not found. Could not print.
Not necesssary to delete file. --> Because not existing
Failed to open file to write config.  --> No error from debug from sd library
...

[greiman]

Can't help since my Nano 33 IoT works with my examples and test programs.

[tickietackie]

I do not think that it has sth. to do with the fact, that I am using an Arduino Nano. That's not the problem. Something of my code around the library is messing up with it. When I use the clean example everything works, also writing. As soon as I want to use the code within my application it fails. I'm going to figure what's causing it ...

[tickietackie]

Ok, so I found the error: I have this class where I declared the SD as a private member and the methods to handle loading and saving:

class SdCardHandler {
 private:
  SdFat SD;

  // static initialized varibale
  static bool initialized;

 public:
  // getter
  bool getInitialized() { return SdCardHandler::initialized; }

  // constructor
  SdCardHandler() {
    // initialize sd card if it is not already
    if (SdCardHandler::initialized == false) {
      // initialize sd card
      SdCardHandler::initialized = this->initializeSdCard();
    }
  }

bool SdCardHandler::initializeSdCard() {
  // Initialize the SD.
  if (!SD.begin(CHIP_SELECT, SPI_SPEED)) {
    SD.initErrorPrint(&Serial);
    DEBUG_LN(F("Failed to initialize SD card."));
    return false;
  } else {
    DEBUG_LN(F("SD card successfully initialized."));
    return true;
  }
}

  bool loadFromFile(const char *filePath, TableConfig &config) {
    File file = SD.open(filePath, FILE_READ);
    //...
  };
  bool loadFromFile(char *filePath, String output){
    File file = SD.open(filePath, FILE_WRITE);
    //...
  };

I think that the variable SD was not available anymore when the object of the class was destroyed and therefore it failed. This meant only the first access to the file worked and this was reading every time. So it was falsely assumed that writing didn't work. Just the first operation on the SD card worked. When declaring SdFat SD; as a global in the cpp file it works ...

[greiman]

SD has the cache and all characteristics of the volume on the card. File structures have a pointer to the instance of SdFat with the file.

So the SdFat instance used to open a file must be in scope to access the file