NebulousLabs / Sia

Blockchain-based marketplace for file storage. Project has moved to GitLab: https://gitlab.com/NebulousLabs/Sia
https://sia.tech
MIT License
2.71k stars 440 forks source link

Support renter upload/download when client and server are separate systems #2100

Open mtlynch opened 7 years ago

mtlynch commented 7 years ago

The following /renter APIs are difficult to use in scenarios where the renter application is not running on the same system as the siad server:

The problem is that the "local" paths in the API are the paths on the siad system rather than the API client system.

As an app developer, I'd like a /renter/upload API where I can give siad the file data and the siapath and Sia just processes the upload. Similarly, I'd like a /renter/download API where I can give siad a siapath and it gives me back the file's data over HTTP, like a normal HTTP download.

Possible workarounds I can think of for now:

lukechampine commented 7 years ago

We have implemented limited support for streaming downloads over HTTP. Uploads are trickier because they force the renter to use a sequential upload algorithm. You also have the problem of resuming; how do you resume an upload that was initiated over HTTP? It's doable but there's a fair amount of logic involved. But on the whole, streaming uploads/downloads is an important feature, and will likely only become more important as the network matures.

avahowell commented 7 years ago

We could upload the entire file in an API request, and have the renter write the file out to disk and keep it locally until the upload is complete. This way, all of the renter's uploading and repair algorithms remain unchanged, the API client simply has to send all of the file data over HTTP to the backend, instead of supplying the path to the file.