pgbackrest host is not getting used for multiple clusters

[kumarashish071092]

I have multiple environments such as dev, qa, and UAT . All the environments have two node clusters but pgbackrest host is shared by all.

When I execute the deploy_cluster.yml playbook, the global is set to repo1-path and it follows for all environments hence only one environment is visible and the rest are giving the below errors:

postgres@pgbackrest:~$ pgbackrest backup --type=full --stanza=supplier-connect-qa
WARN: configuration file contains invalid option 'include-path'
2024-02-15 10:35:37.779 P00   INFO: backup command begin 2.49: --archive-check --no-archive-copy --backup-standby --exec-id=2477007-fbadf106 --log-level-console=info --log-level-file=detail --log-path=/var/log/pgbackrest --pg1-host=10.X.X.X --pg2-host=10.X.X.X --pg1-path=/mnt/Postgres/postgresql/15/supplier-connect-qa/data --pg2-path=/mnt/Postgres/postgresql/15/supplier-connect-qa/data --pg1-port=35432 --pg2-port=35432 --repo1-block --repo1-bundle --repo1-path=/mnt/Postgres/pgbackrest/supplier-connect-dev --repo1-retention-archive=4 --repo1-retention-full=4 --repo1-type=posix --no-resume --stanza=supplier-connect-qa --start-fast --stop-auto --type=full
ERROR: [055]: unable to load info file '/mnt/Postgres/pgbackrest/supplier-connect-dev/backup/supplier-connect-qa/backup.info' or '/mnt/Postgres/pgbackrest/supplier-connect-dev/backup/supplier-connect-qa/backup.info.copy':
       FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-dev/backup/supplier-connect-qa/backup.info' for read
       FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-dev/backup/supplier-connect-qa/backup.info.copy' for read
       HINT: backup.info cannot be opened and is required to perform a backup.
       HINT: has a stanza-create been performed?
2024-02-15 10:35:37.780 P00   INFO: backup command end: aborted with exception [055]

how can I set up my backrest config which can create separate repos for all environments such as :

/mnt/Postgres/pgbackrest/supplier-connect-dev /mnt/Postgres/pgbackrest/supplier-connect-qa /mnt/Postgres/pgbackrest/supplier-connect-uat

[vitabaks]

The repo1-path must be the same in all configurations of the database server and backup repository. I don't see any problems with repo1-path being for example /mnt/data/pgbackrest

Backups will be distributed to different directories according to the stanza name, so it is enough to have a unique stanza name for each cluster (see pgbackrest_stanza variable).

[vitabaks]

I usually use different pgbackrest servers for different environments.

[kumarashish071092]

Hi Vitaliy,

I am attaching my pgbackrest configurations and parameters. I have tried all the options but still getting errors . The error with the attached configuration is below :

***@***.***:/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev$ pgbackrest backup --type=full --stanza=supplier-connect-dev
WARN: configuration file contains invalid option 'include-path'
2024-02-16 06:20:34.253 P00   INFO: backup command begin 2.49: --archive-check --no-archive-copy --backup-standby --exec-id=2665452-851c74ab --log-level-console=info --log-level-file=detail --log-path=/var/log/pgbackrest --pg1-host=10.X.X.X --pg2-host=10.X.X.X --pg1-path=/mnt/Postgres/postgresql/15/supplier-connect-dev/data --pg2-path=/mnt/Postgres/postgresql/15/supplier-connect-dev/data --pg1-port=35432 --pg2-port=35432 --repo1-block --repo1-bundle --repo1-path=/mnt/Postgres/pgbackrest/supplier-connect-qa --repo1-retention-archive=4 --repo1-retention-full=4 --repo1-type=posix --no-resume --stanza=supplier-connect-dev --start-fast --stop-auto --type=full
ERROR: [055]: unable to load info file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info' or '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info.copy':
       FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info' for read
       FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info.copy' for read
       HINT: backup.info cannot be opened and is required to perform a backup.
       HINT: has a stanza-create been performed?
2024-02-16 06:20:34.254 P00   INFO: backup command end: aborted with exception [055]
***@***.***:/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev$

why it is looking for supplier-connect-dev stanza info in supplier-connect-qa directory? Also repo1-path is pointing to QA directory.

what I am missing here?

Regards, Ashish

---

From: Vitaliy Kukharik @.> Sent: Friday, February 16, 2024 3:33 AM To: vitabaks/postgresql_cluster @.> Cc: Kumar Ashish @.>; Author @.> Subject: Re: [vitabaks/postgresql_cluster] pgbackrest host is not getting used for multiple clusters (Issue #576)

[Email from a non-Nagarro source: please exercise caution with links and attachments]

The repo1-path must be the same in all configurations of the database server and backup repository. I don't see any problems with repo1-path being for example /mnt/data/pgbackrest

Backups will be distributed to different directories according to the stanza name, so it is enough to have a unique stanza name for each cluster.

— Reply to this email directly, view it on GitHubhttps://github.com/vitabaks/postgresql_cluster/issues/576#issuecomment-1947411755, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVXUU3VDD3JDHDATQAHBDDTYT2AZXAVCNFSM6AAAAABDKPNZ2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBXGQYTCNZVGU. You are receiving this because you authored the thread.Message ID: @.***>

pgbackrest_install: true # or 'true' pgbackrest_install_from_pgdg_repo: true # or 'false' pgbackrest_stanza: "{{ patroni_cluster_name }}" # specify your --stanza pgbackrest_repo_type: "posix" # or "s3", "gcs", "azure" pgbackrest_repo_host: "10.10.9.122" # dedicated repository host (optional) pgbackrest_repo_user: "postgres" pgbackrest_conf_file: "/etc/pgbackrest/pgbackrest.conf"

see more options https://pgbackrest.org/configuration.html

pgbackrest_conf: global: # [global] section

• { option: "log-level-file", value: "detail" }
• { option: "log-path", value: "/var/log/pgbackrest" }
• { option: "repo1-host", value: "{{ pgbackrest_repo_host }}" }
• { option: "repo1-host-user", value: "{{ pgbackrest_repo_user }}" }
• { option: "repo1-type", value: "{{ pgbackrest_repo_type | lower }}" }
• { option: "repo1-path", value: "/mnt/Postgres/pgbackrest/{{ patroni_cluster_name }}" }
• { option: "repo1-retention-full", value: "4" }
• { option: "repo1-retention-archive", value: "4" }
• { option: "start-fast", value: "y" }
• { option: "stop-auto", value: "y" }
• { option: "resume", value: "n" }
• { option: "link-all", value: "y" }

  - { option: "", value: "" }

  stanza: # [stanza_name] section

• { option: "pg1-path", value: "{{ postgresql_data_dir }}" }
• { option: "recovery-option", value: "recovery_target_action=promote" }
• { option: "log-level-console", value: "info" }
• { option: "process-max", value: "4" }
• { option: "repo1-path", value: "/mnt/Postgres/pgbackrest/{{ patroni_cluster_name }}" }

  - { option: "", value: "" }

  (optional) dedicated backup server config (if "repo_host" is set)

  pgbackrest_server_conf: global:

• { option: "log-level-file", value: "detail" }
• { option: "log-level-console", value: "info" }
• { option: "log-path", value: "/var/log/pgbackrest" }
• { option: "repo1-type", value: "{{ pgbackrest_repo_type | lower }}" }
• { option: "repo1-path", value: "/mnt/Postgres/pgbackrest/{{ patroni_cluster_name }}" }
• { option: "repo1-retention-full", value: "4" }
• { option: "repo1-retention-archive", value: "4" }
• { option: "repo1-bundle", value: "y" }
• { option: "repo1-block", value: "y" }
• { option: "start-fast", value: "y" }
• { option: "stop-auto", value: "y" }
• { option: "resume", value: "n" }
• { option: "link-all", value: "y" }
• { option: "archive-check", value: "y" }
• { option: "archive-copy", value: "n" }
• { option: "backup-standby", value: "y" }

  - { option: "port", value: "{{ postgresql_port }}" }

  the stanza section will be generated automatically

pgbackrest_archive_command: "pgbackrest --stanza={{ pgbackrest_stanza }} archive-push %p"

[kumarashish071092]

It worked with below config in both pgbackrest_server_conf and pgbackrest-conf for global 😂 repo1-path : '/mnt/postgres/pgbackrest'

removing cluster name form repo1-path solved the issue.

We can close this now .

Thanks , Ashish

---

From: Kumar Ashish @.> Sent: Friday, February 16, 2024 4:57 PM To: vitabaks/postgresql_cluster @.>; vitabaks/postgresql_cluster @.> Cc: Author @.> Subject: Re: [vitabaks/postgresql_cluster] pgbackrest host is not getting used for multiple clusters (Issue #576)

Hi Vitaliy,

I am attaching my pgbackrest configurations and parameters. I have tried all the options but still getting errors . The error with the attached configuration is below :

@.:/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev$ pgbackrest backup --type=full --stanza=supplier-connect-dev WARN: configuration file contains invalid option 'include-path' 2024-02-16 06:20:34.253 P00 INFO: backup command begin 2.49: --archive-check --no-archive-copy --backup-standby --exec-id=2665452-851c74ab --log-level-console=info --log-level-file=detail --log-path=/var/log/pgbackrest --pg1-host=10.X.X.X --pg2-host=10.X.X.X --pg1-path=/mnt/Postgres/postgresql/15/supplier-connect-dev/data --pg2-path=/mnt/Postgres/postgresql/15/supplier-connect-dev/data --pg1-port=35432 --pg2-port=35432 --repo1-block --repo1-bundle --repo1-path=/mnt/Postgres/pgbackrest/supplier-connect-qa --repo1-retention-archive=4 --repo1-retention-full=4 --repo1-type=posix --no-resume --stanza=supplier-connect-dev --start-fast --stop-auto --type=full ERROR: [055]: unable to load info file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info' or '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info.copy': FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info' for read FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info.copy' for read HINT: backup.info cannot be opened and is required to perform a backup. HINT: has a stanza-create been performed? 2024-02-16 06:20:34.254 P00 INFO: backup command end: aborted with exception [055] @.:/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev$

why it is looking for supplier-connect-dev stanza info in supplier-connect-qa directory? Also repo1-path is pointing to QA directory.

what I am missing here?

Regards, Ashish

---

From: Vitaliy Kukharik @.> Sent: Friday, February 16, 2024 3:33 AM To: vitabaks/postgresql_cluster @.> Cc: Kumar Ashish @.>; Author @.> Subject: Re: [vitabaks/postgresql_cluster] pgbackrest host is not getting used for multiple clusters (Issue #576)

[Email from a non-Nagarro source: please exercise caution with links and attachments]

The repo1-path must be the same in all configurations of the database server and backup repository. I don't see any problems with repo1-path being for example /mnt/data/pgbackrest

Backups will be distributed to different directories according to the stanza name, so it is enough to have a unique stanza name for each cluster.

— Reply to this email directly, view it on GitHubhttps://github.com/vitabaks/postgresql_cluster/issues/576#issuecomment-1947411755, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVXUU3VDD3JDHDATQAHBDDTYT2AZXAVCNFSM6AAAAABDKPNZ2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBXGQYTCNZVGU. You are receiving this because you authored the thread.Message ID: @.***>