search

Klowner / tussle

Tus daemon written in Typescript using a pluggable architecture
MIT License
37 stars 2 forks source link

How can I get all uploaded file list from storage-r2? #10

Closed BLEE0408 closed 1 year ago

BLEE0408 commented 1 year ago

Hello I want to get all file info from storage-r2 https://github.com/Klowner/tussle/blob/main/examples/cloudflare-worker-r2/src/worker.ts How can I do this?

Klowner commented 1 year ago

Listing storage-r2 files is a little tricky since files are stored as individual chunks in separate R2 records and reassembled by the R2File (which can be obtained through storage.getFile(location) (R2 multipart upload didn't yet exist when storage-r2 was written).

Ideally, during "before-create" or "after-complete" hooks, you would store the location of the uploaded file elsewhere (either in KV or some external service) for later retrieval by their individual locations.

But if you really want to list them from R2, you'll need to use the R2 workers API directly. The downside is that (as previously mentioned), the individual chunks are stored in consecutively numbered files.

location R2 record paths
my/images/cat.png my/images/cat.png/0000000000
my/images/cat.png/0000000001
my/images/cat.png/0000000002

So, if you filter list results to look for any paths ending with 10 zeroes, then stripping the trailing "/0000000000" will give you a location that you can pass to storage.getFile().

BLEE0408 commented 1 year ago

Can I implement this in worker? And can i use this package to solve this problem?: https://github.com/Klowner/tussle/ And Plz provide example for me

Klowner commented 1 year ago

Can I implement this in worker?

This can be implemented in worker, yes.

You can find examples of listing R2 bucket files in the R2 section of the workers API documentation at Cloudflare: https://developers.cloudflare.com/r2/data-access/workers-api/workers-api-reference/#bucket-method-definitions

The Cloudflare Developers discord is also a good resource for information: https://discord.com/invite/cloudflaredev

Klowner commented 1 year ago

Version 0.7.0 adds auto-merge support, so completed uploads should now leave an uploaded file as a single R2 record (unless skipMerge: true is passed to the constructor of TussleStorageR2).