I will shortly be add MTP (Media Transfer Protocol) USB-Device (MTP responder) support to TinyUSB to allow a USB Host to transparently access (read/write) LittleFS as a remotely mounted file system, and am looking for some design assistance of how to achieve this integration most efficiently and securely with LittleFS.
The MTP interface which is widely used to interface Digital Cameras and Android Smart Phones, tablets and MP3 players with host PCs allows a stable, wear-leveling file system to be installed in the internal Flash on an embedded device such as the Raspberry Pi Pico, and provide a much safer file-level interface with a USB Host compared to accessing a FAT file system exposed as a USB Mass Storage Device, with all the synchronization, integrity and Flash wear issues that it entails. While wear leveling could be added to a FAT file system by introducing a wear-leveling layer between the raw NOR/NAND Flash and the FAT file system (example: ESPRESSIF Wear Levelling API) this does not solve the other problems of using a block-level interface file system that has not been designed with Flash storage in mind.
MTP Object Handles
However, MTP requires unique and session-stable 32-bit IDs (Object Handles) to be maintained for each file system object (i.e. files and directories). I am looking into ways to obtain/generate such unique file system object IDs when integrating to a LittleFS file system, while maintaining the LittleFS design principle of bounded memory usage.
Generating MTP Object Handles
Existing MTP responder implementations, e.g. MTP_Teensy, have chosen to scan the file system when an MTP initiator mounts the it and store ID mappings in an index file in the file system root. While this is compatible with any (writable) file system, it is neither elegant, nor very efficient, and causes writes to Flash even when just reading the mounted file system.
Maintaining the ID to file system object mappings in RAM, would be efficient, but neither scalable nor memory bounded.
My current plan for generating MTP Object ID is:
Using the Metadata LFS_TYPE_NAME.tag.id value associated with the file/directory.
Generating and persisting unique 32-bit IDs for each LittleFS file/directory in an associated LFS_TYPE_USERATTR attribute.
Questions
After reviewing the LittleFS DESIGN and SPEC documents, I am uncertain of the uniqueness and stability of the LFS_TYPE_NAME.tag.id property:
Is LFS_TYPE_NAME.tag.id value unique to the parent directory, the metadata block or the file system (implying a 2^10-1 = 1023 file limit)?
If LFS_TYPE_NAME.tag.id is directory- or block-unique, could a globally unique 32-bit MTP object ID be generated from a combination of the LFS_TYPE_NAME.id and its parent LFS_TYPE_DIR.tag.id?
Is the LFS_TYPE_NAME.tag.id value stable across file/directory renaming and move/relocate operations?
The following quote from LFS_TYPE_NAME seems to indicate otherwise:
... the name tag ... can not be reassigned without deleting the file.
Any other recommendations for maintaining unique MTP Object IDs?
Media Transfer Protocol specification
Excerpts from the Media Transfer Protocol specification, version 1.1, April 6th, 2011:
3.4 Object Handles
Object handles are 32-bit identifiers that provide a device- and session-unique consistent reference to a logical object on a device. All handles are represented using the UINT32 simple type. There is no special meaning attached to the value of object handles; they may be chosen in any manner that facilitates device implementation.
Object handles are used in MTP transactions to reference a logical object on the device, but do not necessarily reference actual data constructs on the device. As identified in section 1.3, objects may be exposed through MTP that will be created on-demand. Object handles are only persistent within an MTP session; once a session has been re-opened, all previous values shall be assumed to be invalid, and the contents of the Responder must be re-enumerated if object handles are needed.
The values "0xFFFFFFFF" and "0x00000000" have special meaning and shall not be assigned to objects. The meanings of those values are context-specific.
3.4.1 Assigning Object Handles
Object Handles are assigned by the responder. They must be globally unique across all storages on the device.
Object Handles are only defined within an open session. If the Initiator has not yet opened a session, object handles do not have any meaning and cannot be used. When a session is closed, either intentionally by the initiator or as a result of an error or USB interruption, all Object Handles are invalidated and must be re-acquired by the Initiator. Object Handles do not persist between sessions.
If an object is deleted in a session, the Responder shall not re-use the deleted object’s Object Handle in the same session.
MWP
commented
2 weeks ago
I will shortly be add MTP (Media Transfer Protocol) USB-Device (MTP responder) support to TinyUSB to allow a USB Host to transparently access (read/write) LittleFS as a remotely mounted file system, and am looking for some design assistance of how to achieve this integration most efficiently and securely with LittleFS.
The MTP interface which is widely used to interface Digital Cameras and Android Smart Phones, tablets and MP3 players with host PCs allows a stable, wear-leveling file system to be installed in the internal Flash on an embedded device such as the Raspberry Pi Pico, and provide a much safer file-level interface with a USB Host compared to accessing a FAT file system exposed as a USB Mass Storage Device, with all the synchronization, integrity and Flash wear issues that it entails. While wear leveling could be added to a FAT file system by introducing a wear-leveling layer between the raw NOR/NAND Flash and the FAT file system (example: ESPRESSIF Wear Levelling API) this does not solve the other problems of using a block-level interface file system that has not been designed with Flash storage in mind.
MTP Object Handles
However, MTP requires unique and session-stable 32-bit IDs (Object Handles) to be maintained for each file system object (i.e. files and directories). I am looking into ways to obtain/generate such unique file system object IDs when integrating to a LittleFS file system, while maintaining the LittleFS design principle of bounded memory usage.
Generating MTP Object Handles
Existing MTP responder implementations, e.g. MTP_Teensy, have chosen to scan the file system when an MTP initiator mounts the it and store ID mappings in an index file in the file system root. While this is compatible with any (writable) file system, it is neither elegant, nor very efficient, and causes writes to Flash even when just reading the mounted file system.
Maintaining the ID to file system object mappings in RAM, would be efficient, but neither scalable nor memory bounded.
My current plan for generating MTP Object ID is:
LFS_TYPE_NAME.tag.idvalue associated with the file/directory.LFS_TYPE_USERATTRattribute.Questions
After reviewing the LittleFS DESIGN and SPEC documents, I am uncertain of the uniqueness and stability of the
LFS_TYPE_NAME.tag.idproperty:LFS_TYPE_NAME.tag.idvalue unique to the parent directory, the metadata block or the file system (implying a 2^10-1 = 1023 file limit)?LFS_TYPE_NAME.tag.idis directory- or block-unique, could a globally unique 32-bit MTP object ID be generated from a combination of theLFS_TYPE_NAME.idand its parentLFS_TYPE_DIR.tag.id?LFS_TYPE_NAME.tag.idvalue stable across file/directory renaming and move/relocate operations? The following quote fromLFS_TYPE_NAMEseems to indicate otherwise:Media Transfer Protocol specification
Excerpts from the Media Transfer Protocol specification, version 1.1, April 6th, 2011: