supabase / storage

S3 compatible object storage service that stores metadata in Postgres
https://supabase.com/docs/guides/storage
Apache License 2.0
760 stars 107 forks source link

Downloading or backing up entire storage bucket #363

Open kurtdicus opened 1 year ago

kurtdicus commented 1 year ago

Improve documentation for Storage API while awaiting longer term solution for being able to download all files from storage buckets

User Problem

User would like to download all files from a storage bucket more easily. User would like to backup storage buckets more easily.

Potential Solutions

  1. Client-side code using current API to list the contents of a bucket, downloading each file asyncronously, also using the API (see simple Node example below).
  2. Supabase could provide a new endpoint called something like .storage.from('mybucket').downloadAll() or .storage.from('mybucket').backup(). Future version request for options like {asSingleZipFile: true, callHookWhenComplete: 'https://myhook_to_call_when_complete', etc: 'etc'}.
  3. Alternatively, Supabase could provide a new endpoint called something like .storage.from('mybucket').createBackupFile('backups') which does the backend work to create a new file which gets placed into 'backups' storage bucket. Finally, the current .download() API endpoint could be used to get the backup file.

Simple example Node script for looping through entire storage folder and downloading each file asynchronously. getBucketToLocalPath

HighPriest commented 3 weeks ago

Extending this request with "ListAll" endpoint, for listing all files with their full paths in a bucket. https://github.com/supabase-community/storage-csharp/issues/18

Thank you 🥰