skyplane-project / skyplane

🔥 Blazing fast bulk data transfers between any cloud 🔥
https://skyplane.org
Apache License 2.0
1k stars 58 forks source link

Add back compression and encryption #877

Closed lynnliu030 closed 1 year ago

sarahwooders commented 1 year ago

So in the current implementation, my understanding is that each time data moves between a VM it is compressed and decompressed? This is fine for now, but maybe we should add an issue so in the future the data is compressed/encrypted just once when it is read from the object store.

lynnliu030 commented 1 year ago

So in the current implementation, my understanding is that each time data moves between a VM it is compressed and decompressed? This is fine for now, but maybe we should add an issue so in the future the data is compressed/encrypted just once when it is read from the object store.

@sarahwooders I don't think so? all of these are specified in the gateway program; it'll only be decompressed or decrypted if you set it explicitly in the gateway programs of VMs located in destination regions