rec / hardback

📓 Hardcopy backups of digital data 📓
MIT License
1 stars 0 forks source link

Use document-level error correction #1

Open rec opened 5 years ago

rec commented 5 years ago

The QR code format provides some level of error correction (using Reed-Solomon codes) but if a whole QR code is missing, there is no way to reconstruct it.

Using a higher-level error correction code on the data would allow whole QR codes to be reconstructed in case of damage.

We'd have to divide the data into groups consisting of many QR codes. We already have blocks (of the original file) and chunks (a block, plus header information) so these would be "QR groups". Then further QR code would contain the check data.

Reed-Solomon codes seem like the logical solution, because they handle corrections "symbol at a time" but also because they can handle twice as many erasures (positions which you know are missing) as errors, and with a physical book, erasures are much more likely.

Unfortunately, the leading Python RS code library has frozen development, though it seems to work. A fork has some improvements but went inactive almost two years ago.

And there are others, or an interface to a C++ library could be written.

Worth attempting but later. We would store it as metadata blocks...