charliemirabile
commented
1 month ago We kind of attempt to deal with this in the smtp code https://github.com/underground-software/singularity/blob/e33834f31ed694139a614aa998b71db76e64c951/smtp/smtp.c#L115 but this is not a complete solution because it does not deal with differences that can exist even between little endian hosts, and it means that the code simply does not compile on big endian hosts at all
Currently the email code deals with some binary data: the xattrs storing file offsets and the email structs in the journal. These are stored as binary data in whatever the native format and byte ordering for the host machine is. This means that it is not possible to migrate data between instances which differ in their ABI.
We should at least document this limitation, or if we want to fix it pick a specific layout for all of this data including byte ordering and more explicitly corral the data in ram into and out of the storage format before storing on disk. If we design the format well, it might be possible to make this corralling a formality and basically a no-op for at least one of big or little endian hosts.
We might want to make it possible select between big and little endian byte ordering for the storage format at build time with the understanding that data will not be portable between instances compiled with differing configurations, but offering the increased performance that comes with matching the storage to the host format if all instances share the same byte ordering regardless of which it is.