search

EMCECS / ecs-sync

ecs-sync is a bulk copy utility that can move data between various systems in parallel
Apache License 2.0
61 stars 22 forks source link

Add Source/Target Support type: Device or PIPE #5

Closed victorock closed 8 years ago

victorock commented 8 years ago

Hi, It would be really amazing if we could have support for block devices or PIPE. If so, we would be able to take FileSystem Snapshots and send/receive it from ECS, or facilitate data manipulation without any requirement to save it in a filesystem (consuming precious space) before sync.

Sincerely, Victor da Costa

twincitiesguy commented 8 years ago

Unfortunately this is out of scope for the project. However, you could easily do this with the AWS CLI if ECS S3 is your target. Something like the following should work:

sudo dd if=/dev/sda1 | gzip --fast | aws s3 cp - s3://my-bucket/sda1-snapshot-2016-02-05 (source)

Since it must be single-threaded, there's no real advantage to adding a stream source.

Note: to get the AWS CLI to work with ECS, you must configure it to use v2 auth.