vitabaks / postgresql_cluster

PostgreSQL High-Availability Cluster (based on Patroni). Automating with Ansible.
https://postgresql-cluster.org
MIT License
1.48k stars 394 forks source link

Failed to find any configured storage #696

Closed algoritmsystems closed 1 month ago

algoritmsystems commented 1 month ago

I depoloyed postgresql cluster and during installion the database with initdb was deployed to my minio server, but after it can't find any configured storage:

wal-g backup-push /var/lib/postgresql/14/main/ ERROR: 2024/07/13 18:09:47.369761 Failed to find any configured storage

main.yml config is:

wal_g_install: true  
wal_g_version: "3.0.2"
wal_g_path: "/usr/local/bin/wal-g"
wal_g_json:  # config https://github.com/wal-g/wal-g#configuration
  - { option: "AWS_ACCESS_KEY_ID", value: "myminioadmin" }
  - { option: "AWS_SECRET_ACCESS_KEY", value: "myminiopass" } 
  - { option: "WALG_S3_PREFIX", value: "s3://mnt/" } 
  - { option: "WALG_COMPRESSION_METHOD", value: "brotli" }   
  - { option: "AWS_ENDPOINT", value: "http://172.20.20.26:9000" }  
  - { option: "AWS_S3_FORCE_PATH_STYLE", value: "true" } 
  - { option: "PGDATA", value: "{{ postgresql_data_dir }}" }
  - { option: "PGHOST", value: "{{ postgresql_unix_socket_dir }}" }
  - { option: "PGPORT", value: "{{ postgresql_port }}" }
  - { option: "PGUSER", value: "{{ patroni_superuser_username }}" }
algoritmsystems commented 1 month ago

After creation wal-g.yml file it is working:

WALG_S3_PREFIX: s3://mnt/data/ AWS_ACCESS_KEY_ID: myminioadmin AWS_SECRET_ACCESS_KEY: myminiopass AWS_ENDPOINT: http://172.20.20.26:9000 AWS_S3_FORCE_PATH_STYLE: "true" WALG_COMPRESSION_METHOD: brotli PGHOST: 127.0.0.1 PGPORT: 5432 PGUSER: postgres PGPASSWORD: postgrespass

wal-g --config wal-g.yaml backup-push /var/lib/postgresql/14/main/

but synch/asynch backup isn't working automaticly:

systemctl status patroni ● patroni.service - Runners to orchestrate a high-availability PostgreSQL - Patroni Loaded: loaded (/etc/systemd/system/patroni.service; enabled; preset: enabled) Active: active (running) since Sat 2024-07-13 19:21:01 +05; 4s ago Process: 830079 ExecStartPre=/usr/bin/sudo /sbin/modprobe softdog (code=exited, status=0/SUCCESS) Process: 830081 ExecStartPre=/usr/bin/sudo /bin/chown postgres /dev/watchdog (code=exited, status=0/SUCCESS) Main PID: 830084 (patroni) Tasks: 9 (limit: 19134) Memory: 377.3M CPU: 3.998s CGroup: /system.slice/patroni.service ├─830084 /usr/bin/python3 /usr/local/bin/patroni /etc/patroni/patroni.yml ├─830114 /usr/lib/postgresql/14/bin/pg_basebackup --pgdata=/var/lib/postgresql/14/main -X stream "--dbname=dbname=postgres user=replicator host=172.20.20.24 port=5432" --max-rate=100M --checkpoint=fast └─830132 /usr/lib/postgresql/14/bin/pg_basebackup --pgdata=/var/lib/postgresql/14/main -X stream "--dbname=dbname=postgres user=replicator host=172.20.20.24 port=5432" --max-rate=100M --checkpoint=fast Jul 13 19:21:01 pdb20i23c1 sudo[830081]: pam_unix(sudo:session): session closed for user root Jul 13 19:21:01 pdb20i23c1 systemd[1]: Started patroni.service - Runners to orchestrate a high-availability PostgreSQL - Patroni. Jul 13 19:21:01 pdb20i23c1 patroni[830084]: 2024-07-13 19:21:01,743 INFO: Selected new etcd server http://172.20.20.24:2379 Jul 13 19:21:01 pdb20i23c1 patroni[830084]: 2024-07-13 19:21:01,763 INFO: No PostgreSQL configuration items changed, nothing to reload. Jul 13 19:21:01 pdb20i23c1 patroni[830084]: 2024-07-13 19:21:01,823 INFO: Lock owner: pdb20i24c2; I am pdb20i23c1 Jul 13 19:21:01 pdb20i23c1 patroni[830084]: 2024-07-13 19:21:01,874 INFO: trying to bootstrap from leader 'pdb20i24c2' Jul 13 19:21:01 pdb20i23c1 patroni[830097]: INFO: 2024/07/13 19:21:01.996186 Selecting the latest backup... Jul 13 19:21:01 pdb20i23c1 patroni[830097]: INFO: 2024/07/13 19:21:01.999041 Backup to fetch will be searched in storages: [default] Jul 13 19:21:02 pdb20i23c1 patroni[830097]: ERROR: 2024/07/13 19:21:02.007451 Failed to select backup: No backups found Jul 13 19:21:02 pdb20i23c1 patroni[830084]: 2024-07-13 19:21:02,022 ERROR: Error creating replica using method wal_g: /usr/local/bin/wal-g backup-fetch /var/lib/postgresql/14/main LATEST exited with code=1

ll /var/lib/postgresql/.walg.json

-rw-r--r-- 1 postgres postgres 359 Jul 13 17:39 /var/lib/postgresql/.walg.json cat /var/lib/postgresql/.walg.json { "AWS_ACCESS_KEY_ID": "myminioadmin", "AWS_SECRET_ACCESS_KEY": "myminiopass", "WALG_S3_PREFIX": "s3://mnt/", "WALG_COMPRESSION_METHOD": "brotli", "AWS_ENDPOINT": "http://172.20.20.26:9000", "AWS_S3_FORCE_PATH_STYLE": "true", "PGDATA": "/var/lib/postgresql/14/main", "PGHOST": "/var/run/postgresql", "PGPORT": "5432", "PGUSER": "postgres" }

vitabaks commented 1 month ago

This is strange, perhaps the latest version of wal-g requires you to explicitly specify the path to the configuration.

Try it this:

wal_g_path: "/usr/local/bin/wal-g --config /var/lib/postgresql/.walg.json"
algoritmsystems commented 1 month ago

Thanks @vitabaks also I configured pg_hba and minio. It is working! Thank you!