Klowner / tussle

Tus daemon written in Typescript using a pluggable architecture
MIT License
37 stars 2 forks source link

Cloudflare R2 example #5

Closed phpb-com closed 1 year ago

phpb-com commented 1 year ago

https://github.com/Klowner/tussle/tree/main/examples/cloudflare-worker-r2 works for upload but how does one suppose to reassemble the file for reading/processing/downloading ? Despite Readme stating that files are available for download after uploading, I could not find any meaningful way to do it. Thanks.

Klowner commented 1 year ago

Good question, I need better documentation.

TussleStorageR2 has a getFile() method which returns an R2File instance.

R2File has a body property which exposes a ReadableStream which handles reassembling the parts in correct order and returning them as a single stream.

If you have thousands of file parts, you will likely need to get a little more creative since you'll probably exhaust the Cloudflare Worker's CPU limits or R2 API call limits, in which case you can devise your own implementation by iterating over the R2File's keys array (which is just an ordered array of R2 object keys representing the file parts).

phpb-com commented 1 year ago

I really appreciate your prompt reply. Let me dig into it a little more and see what I can do. In the end, I think I would need to go through the S3 backend so I can have the object reassembled into one. I hope for R2 worker bindings to support multipart soon.

Klowner commented 1 year ago

You could always add an after-complete handler that stores the R2File.body as a new single R2 object and then delete the keys.

On Fri, Sep 23, 2022, 6:33 PM phpb-com @.***> wrote:

Closed #5 https://github.com/Klowner/tussle/issues/5 as completed.

— Reply to this email directly, view it on GitHub https://github.com/Klowner/tussle/issues/5#event-7450689502, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACJHTHF4FQEPSF5FMXYALV7YV3LANCNFSM6AAAAAAQTFVK6M . You are receiving this because you commented.Message ID: @.***>

phpb-com commented 1 year ago

True, but that gets very expensive very quickly, so it is overall cheaper to buffet through other S3 like service