Data deduplication with erasure encoding

[MeijeSibbel]

Combine erasure coding with data deduplication to simultaneously reduce the overall redundancy in data while increasing the redundancy of unique data. Deduplication also requires less network transfer.

http://shiftleft.com/mirrors/www.hpl.hp.com/personal/Mark_Lillibridge/Reliability/website_draft.pdf https://en.wikipedia.org/wiki/Data_deduplication

[braydonf]

Any compression would need to be before encryption, as compressing encrypted data will have little benefit. Right now this is handled by zipping/tar-ing before uploading.