softlayer / softlayer-object-storage-backup

Other
11 stars 3 forks source link

RFC: Local compression #17

Closed CrackerJackMack closed 11 years ago

CrackerJackMack commented 11 years ago

Just as in #16 we can use chunked compression too. this method looks much easier to impliment as we wouldn't have to impliment a key strategy and you'd want to compress before encrypting anyway.

At any rate, I think we'll still end up storing the original size and original md5 in meta-data. Which also means that even without checksumming enabled, we'll have to compute each chunk we upload regardless.

So enabling compression and/or encryption will have a heavy CPU penality associated with it which in turn will affect upload speeds.