supabase / cli

Supabase CLI. Manage postgres migrations, run Supabase locally, deploy edge functions. Postgres backups. Generating types from your database schema.
https://supabase.com/docs/reference/cli/about
MIT License
1.09k stars 212 forks source link

Add support for different Storage Backend during local development #2650

Open AravindPrabhs opened 2 months ago

AravindPrabhs commented 2 months ago

Is your feature request related to a problem? Please describe.

I would like to use a S3 Compatible Storage Backend during local development and configure it using config.toml.

Describe the solution you'd like

I would like to be able to specify the following env variables to the storage api server:


STORAGE_BACKEND=s3

#######################################
# S3 Backend
#######################################
STORAGE_S3_BUCKET=supa-storage-bucket
STORAGE_S3_MAX_SOCKETS=200
STORAGE_S3_ENDPOINT=http://127.0.0.1:9230
STORAGE_S3_FORCE_PATH_STYLE=true
STORAGE_S3_REGION=local

# unsure if these are right but don't think connection is being made to minio anyway
AWS_ACCESS_KEY_ID=minioadmin 
AWS_SECRET_ACCESS_KEY=minioadmin

Though, separate I wonder if it possible to have a more general API to pass/pipe in whatever environment variables are specified in the config.toml for the particular service so it is easier to try out individual feature changes without forking the CLI.

Describe alternatives you've considered

My original problem is the file storage backend is not performant for range requests on large files please see my original discussion in the storage repo (https://github.com/supabase/storage/discussions/540).

AravindPrabhs commented 2 months ago

I managed to connect the minIO docker instance by adding environment variables field to the storage struct.

// pkg/config/config.io
storage struct {
...
EnvironmentVariables map[string]string    `toml:"env_environment_variables,omitempty"`
}

and then modified start.go so the environment variables are passed to the container

for key, value := range utils.Config.Storage.EnvironmentVariables {
   envVars = append(envVars, fmt.Sprintf("%s=%s", key, value))
}

happy to put in a pull request along these lines, and would this approach be worth extending for other services ?