search

skyplane-project / skyplane

🔥 Blazing fast bulk data transfers between any cloud 🔥
https://skyplane.org
Apache License 2.0
1.08k stars 62 forks source link

[request] Add VM source and destinations #801

Open sarahwooders opened 1 year ago

sarahwooders commented 1 year ago

Please describe problem to be solved
Currently Skyplane only support transferring between object stores. We should add support to transfer data to and from cloud virtual machines.

Suggest a solution

API Modification We will need to add a IP address specified endpoint to Skyplane

client.copy("src-up-addr://home/user/file", "dest-ip-addr://home/user/file")

For now, we can assume the the Skyplane docker container is running on the specified IP.

Gateway Modification We assume that the user needs to start a Skyplane gateway docker container on a machine for it to be a source or destination node. For the destination, the gateway needs to re-assemble the chunk files into the original file structure format in the local directory.