Gateway difficult to use in kubernetes

[kallisti5]

What version of Gateway-ST are you using (gateway version)?

1.6.0

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (uname -mrsv)?

x86_64

Issue

Trying to deploy the gateway-st in a kubernetes environment. getting the following:

2022-02-18T02:26:11.527Z    INFO    Configuration loaded    {"Location": "/root/.local/share/storj/gateway/config.yaml"}
2022-02-18T02:26:11.528Z    INFO    Telemetry enabled   {"instance ID": "36:ff:28:a4:e1:de"}
2022-02-18T02:26:11.528Z    INFO    Starting Storj DCS S3 Gateway

2022-02-18T02:26:11.528Z    INFO    Endpoint: 0.0.0.0:7777

2022-02-18T02:26:11.528Z    INFO    Access key: insecure-dev-access-key

2022-02-18T02:26:11.528Z    INFO    Secret key: insecure-dev-secret-key

ERROR Unable to create directory specified config-dir=/root/.local/share/storj/gateway/minio: mkdir /root/.local/share/storj/gateway/minio: read-only file system

mostly due to the weird config directory layout of the gateway. Secrets are mixed with writable files?

[kallisti5]

Example here:

https://github.com/haiku/infrastructure/blob/master/deployments/cdn.yml

It doesn't even pick up on my minio admin user / password :-|

[amwolff]

So the config directory that it wants to create is created by the underlying MinIO (it will always want to create this directory structure). See explanation. If you have /root/.local/share/storj/gateway/minio/certs/CAs mounted, it shouldn't complain anymore.

mostly due to the weird config directory layout of the gateway. Secrets are mixed with writable files?

It doesn't even pick up on my minio admin user / password :-|

In the way you want to launch Gateway-ST, no secrets will be written to persistent storage (they would only be written if you used the setup sub-command). The reason it doesn't pick up your credentials is that we overwrite them. The proper way to set them is through --minio.access-key and --minio.secret-key flags (or STORJ_MINIO_SECRET_KEY and STORJ_MINIO_SECRET_KEY environmental variables). We probably can also support the usual MinIO way here.

[kallisti5]

/root/.local/share/storj/gateway/minio/certs/CAs

Any chance of it using /tmp or something for these CA's? The directory structure is pretty weird for a service a lot of people are going to run in a container. Unnecessary dedicated PVC in kubernetes are an un-needed expense for users.

or STORJ_MINIO_SECRET_KEY and STORJ_MINIO_SECRET_KEY environmental variables)

STORJ_MINIO_SECRET_KEY / STORJ_MINIO_SECRET_KEY would work fine... they're just not documented anywhere as an option :-)

[amwolff]

Sorry for the late reply!

/root/.local/share/storj/gateway/minio/certs/CAs

Any chance of it using /tmp or something for these CA's? The directory structure is pretty weird for a service a lot of people are going to run in a container. Unnecessary dedicated PVC in kubernetes are an un-needed expense for users.

You can use whatever directory you prefer for these by setting STORJ_MINIO_DIR. However, if you set Access Key ID and Secret Access Key manually, then you don't need to mount anything as this directory won't be used for reading secrets because they are already supplied.

By default, the directory structure is as follows:

• $CONFDIR - for Storj-specific Gateway-ST config file
• $CONFDIR/minio - (STORJ_MINIO_DIR) - for MinIO-specific files

The top-level directory is only used if you configure Gateway-ST using the introductory wizard or create the config file manually.

or STORJ_MINIO_SECRET_KEY and STORJ_MINIO_SECRET_KEY environmental variables)

STORJ_MINIO_SECRET_KEY / STORJ_MINIO_SECRET_KEY would work fine... they're just not documented anywhere as an option :-)

Sorry that you have run into this. I agree we should have documented this behavior and startup options better, and there's an ongoing effort to do this, so I hope this will be resolved soon. Thank you for using Gateway-ST! Let me know if you have any other questions, and otherwise, we can close the issue.