search

zalando / postgres-operator

Postgres operator creates and manages PostgreSQL clusters running in Kubernetes
https://postgres-operator.readthedocs.io/
MIT License
4.35k stars 980 forks source link

chown: changing ownership of '/home/postgres/pgdata/pgroot/data': Operation not permitted when using NFS #2325

Open 4szt4l opened 1 year ago

4szt4l commented 1 year ago

I just tried a minimal cluster:

helm repo add postgres-operator-charts https://opensource.zalando.com/postgres-operator/charts/postgres-operator -n postgres-operator
helm install postgres-operator postgres-operator-charts/postgres-operator -n postgres-operator
curl https://raw.githubusercontent.com/zalando/postgres-operator/master/manifests/minimal-postgres-manifest.yaml | kubectl apply -n postgres-operator -f -

The commands above work fine using EKS in AWS. But when I try to create it in bare-metal k8s, it throws permission errors:

chown: changing ownership of '/home/postgres/pgdata/pgroot/data': Operation not permitted
...
chown: changing ownership of '/home/postgres/pgdata/pgroot': Operation not permitted

I can use the NFS with other postgresql solutions, e.g. with bitnami's helm chart. How could I debug this? I'm using this NFS provisioner without any custom settings (except for setting NFS server IP and path): https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

stevef1uk commented 12 months ago

I have basically the same issue when trying to use an nfs persistent volume I am getting loads of permission errors and CrashLoopBackOff status on the cluster's pods. I set-up my NFS to use postgres as the user and the persistent volumes are curated with ID postgres. I have changed the configmap.yaml to set the following values:

spilo_runasuser: "1001" spilo_runasgroup: "1001" spilo_fsgroup: "1001" spilo_privileged: "true"

(as uid=1001(postgres) gid=1001(postgres) groups=1001(postgres),100(users))

Loads of errors e.g.

ostgres mkdir: cannot create directory ‘/run/certs’: Permission denied postgres /launch.sh: 29: cannot create /run/tmp/passwd: Directory nonexistent postgres /launch.sh: 30: cannot create /etc/passwd: Permission denied postgres rm: cannot remove '/run/tmp/passwd': No such file or directory postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log/postgresql-0.csv': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log/postgresql-7.csv': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log/postgresql-1.csv': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log/postgresql-5.csv': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log/postgresql-2.csv': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log/postgresql-4.csv': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log/postgresql-6.csv': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log/postgresql-3.csv': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/pg_log': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot/data': Operation not permitted postgres chown: changing ownership of '/home/postgres/pgdata/pgroot': Operation not permitted postgres chown: cannot access '/run/certs': No such file or directory postgres chmod: cannot access '/run/tmp': No such file or directory postgres 2023-11-07 10:06:32,552 - bootstrapping - INFO - Figuring out my environment (Google? AWS? Openstack? Local?) postgres 2023-11-07 10:06:34,563 - bootstrapping - INFO - Could not connect to 169.254.169.254, assuming local Docker setup postgres 2023-11-07 10:06:34,565 - bootstrapping - INFO - No meta-data available for this provider postgres 2023-11-07 10:06:34,566 - bootstrapping - INFO - Looks like you are running local postgres 2023-11-07 10:06:34,651 - bootstrapping - INFO - Configuring patroni postgres Traceback (most recent call last): postgres File "/scripts/configure_spilo.py", line 1156, in postgres main() postgres File "/scripts/configure_spilo.py", line 1106, in main postgres write_patroni_config(config, args['force']) postgres File "/scripts/spilo_commons.py", line 85, in write_patroni_config postgres write_file(yaml.dump(config, default_flow_style=False, width=120), PATRONI_CONFIG_FILE, force) postgres File "/scripts/spilo_commons.py", line 74, in write_file postgres with open(filename, 'w') as f: postgres PermissionError: [Errno 13] Permission denied: '/run/postgres.yml'

I should say I tried with the default config first but got an error message that the data files were created by a different owner:

postgres running bootstrap script ... 2023-11-07 10:36:30.869 UTC [180] FATAL: data directory "/home/postgres/pgdata/pgroot/data" has wrong ownership postgres 2023-11-07 10:36:30.869 UTC [180] HINT: The server must be started by the user that owns the data directory. postgres child process exited with exit code 1

On step forward after adding just the line: spilo_fsgroup: "1001" and changing the postgres-operator to use the user postgres

postgres PermissionError: [Errno 13] Permission denied: '/var/run/secrets/kubernetes.io/serviceaccount/token'