SPI Flash driver

[deadprogram]

I'd like to have a driver for SPI flash drives similar to what https://github.com/adafruit/Adafruit_SPIFlash can do. Ideally, it would also support the CircuitPython extensions to the file system for when those are in use.

[aykevl]

That would be useful. When defining such a driver, think of these things:

• I think it would be best to split the filesystem from the block device driver. MicroPython has such a division and it works quite well: it enables other filesystems than just FAT (which is a rather bad filesystem for embedded devices except that it's simple). Maybe we could even use a generic FAT library, which would make testing much easier on a host system (use memory as backend, provide a FUSE interface, and stress-test the filesystem with all the tools that are out there already for testing filesystems).
• The built-in flash of many chips can be used as well, so it would be useful to not have the FAT driver assume it uses SPI flash. In fact, I once got a FAT filesystem working on the nrf51 but it wasn't very usable due to the small amount of available flash on the chip (when using MicroPython).
• There are basically two classes of flash: NAND and NOR. Most embedded devices have NOR flash while most SSDs have NAND flash. The difference is that NOR flash can be written to at a much smaller granularity, often just 4 bytes at a time. Also, page sizes tend to be smaller.
  • Page size varies by chip: the nrf51 has 1kB, nrf52 has 4kB and the flash chip usually shipped with the esp8266 also has a page size of 4kB.
  • An erased flash page is usually all 0xff with newly written data setting bits to 0 (but not back to 1). Depending on the device, you may be able to set more bits in the word to 0 without erasing first. Some devices (IIRC the STM32L series, among others) have the bits reversed.

[bgould]

FYI I'm making decent progress with this... I'm able to access the "internal" SPI flash on the Itsy Bitsy M0 and have implemented functions for doing low level IO operations on the block device. Also I've forked https://github.com/mitchellh/go-fs and with a few tweaks have been able to use that start implementing functionality to read from a FAT filesystem.

@deadprogram I was wondering if you could please clarify what functionality you mean by "the CircuitPython extensions to the file system for when those are in use" - can you point me to any documentation on what those extensions do so that I can think about making accomodations for that as I am working?

[deadprogram]

Great news about your progress!

I was looking and seems like the FAT filesystem used is now standardized? There appears to have been some MBR hacks previously, but looks like they were removed from the Adafruit code.

[bgould]

Updates:

• Low level I/O driver for SPI NOR flash is here: https://github.com/tinygo-org/drivers/pull/124
• I've experimented with 2 filesystem implementation on top of the above driver:
  • https://github.com/bgould/go-fatfs
  • https://github.com/bgould/go-littlefs

Of the two, LittleFS is more full-featured and is recommended for use with SPI flash, as it is far superior with respect to resiliency and wear leveling.

The FAT implementation is currently read-only, but is useful for the following scenarios for instance:

• If you have a board that supports CircuitPython, you can flash CP onto it and then edit files on the USB drive that shows up. You can then flash a TinyGo program to the board and read the files using go-fatfs. Wear leveling is not a concern for read-only usage so this could be a good way to store wifi credentials or other data that you don't want to hardcode into your programs.
• Similarly, if CircuitPython is not supported on your board or if you have a lot of boards to set up, you can build a FAT filesystem image on a computer and then flash directly onto the flash chip using something like this https://www.pomonaelectronics.com/products/test-clips/soic-clip-8-pin via flashrom and some converter like FT232H.
• Eventually if USB mass storage class is made to work, a good setup could be to have the host system have read-write access and have the microcontroller use read-only. This would basically be like the first scenario above, except we would be eliminating the CircuitPython middleman.

I do eventually intend to make the FAT library be read-write, but for the library I used to be effective on SPI NOR flash it will require a caching layer that is not implemented yet. Also if read-write is really necessary, LittleFS is superior in all circumstances except for when it comes to compatibility with a host system (and even in that case, there is a littlefs FUSE module can possibly be utilized).

I will work on preparing documentation on how to wire everything together into a usable setup.

[bgould]

One thing to add... both of the filesystem implementations I mentioned above are designed to be as close as possible to the os package, and both are easy to integrate with https://github.com/tinygo-org/tinygo/pull/1012 if/when that gets included in TinyGo.