Feature: detect file-system email corruption

[haraldrudell]

— it is value if mox can detect and scrub for stored rfc822 that has been corrupted, using sha256 hashes scrub would be in the admin interface, just listing what has been corrupted on reading an email for say FETCH and hash is not matching, — mox could return error and — output filename and expected hash to the log

[mjl-]

i want to encrypt all data stored on disk at some point. the scheme would use AEAD, so all reads would detect corruption of stored data.

i also want to compress data. i'll be looking for a scheme that:

• handles incoming streaming writes (no random writes needed)
• compresses (zstd?)
• encrypts (with an aead algorithm)
• can flush the compression & encryption state at will. e.g. mime part boundaries. should also flush after some max size.
• allows seeking before reading. the common case of seeking to a part (as flushed while writing) should be efficient.

it's been a long time since data i read back from the file system was corrupted. in that case, it was a hard drive returning modified bytes at a specific sector. at least that case would be detected, but not repairable.

but perhaps you're thinking about other kinds of corruption you want to detect?