Add Amazon s3 support for archives service.

[aebruno]

Thanks for this great Nagios plugin. We are currently using pgBackRest with remote storage in Amazon s3 and wanted to use your plugin for monitoring our backups. This PR adds support for listing remote archived WALs stored in Amazon s3 for the archives service. Let me know what you think and happy to make any modifications you see fit.

A new boolean arg --repo-s3 is added to enable s3 support. This requires the --config arg so that we can parse the AWS Access/Secret keys and bucket name which should already be configured if using s3 support in pgBackRest. The --repo-path arg should be set to the path prefix within the s3 bucket.

The following perl modules are required:

• Net::Amazon::S3
• Config::IniFiles

These can be easily installed on debian/ubuntu systems with:

$ apt-get install libnet-amazon-s3-perl libconfig-inifiles-perl

Example usage:

$ check_pgbackrest --stanza=demo --service=archives --format=human \
      --repo-s3 --repo-path /pgbackrest/archive \
      --config /etc/pgbackrest.conf

[dwsteele]

I think we should work to get the information that check_pgbackrest needs into the pgbackrest info output. As we add new storage drivers it will be a challenge to keep all of them working in this project.

[pgstef]

Hi,

I agree with David, we should focus on implementing all the "data collection" part in pgBackRest itself to avoid multiplying data access methods here.

In the mean time, I tried your script upon my test setup (https://github.com/dalibo/check_pgbackrest/commit/ad43f4eb5c843f7adbe8c8df50f805fc1b35634e) using MinIO.

I unfortunately can't make it work easily.

pgbackrest.conf:

[global]
repo1-path=/repo1
repo1-type=s3
repo1-s3-endpoint=minio.local
repo1-s3-bucket=pgbackrest
repo1-s3-verify-ssl=n
repo1-s3-key=***
repo1-s3-key-secret=***
repo1-s3-region=eu-west-3

Output:

# /check_pgbackrest/check_pgbackrest_tmp --stanza=my_stanza --service=archives --repo-path=/repo1 --repo-s3 --config=/etc/pgbackrest.conf --debug
DEBUG: archives_dir: /repo1/my_stanza/11-1
DEBUG: repo1_bucket: pgbackrest
DEBUG: repo1-s3-endpoint: minio.local
Internal Server Error at /usr/share/perl5/vendor_perl/Net/Amazon/S3/Client.pm line 106.
    Net::Amazon::S3::Client::_send_request('Net::Amazon::S3::Client=HASH(0x281a0e8)', 'HTTP::Request=HASH(0x442c8b8)', undef) called at /usr/share/perl5/vendor_perl/Net/Amazon/S3/Client.pm line 120
    Net::Amazon::S3::Client::_send_request_xpc('Net::Amazon::S3::Client=HASH(0x281a0e8)', 'HTTP::Request=HASH(0x442c8b8)') called at /usr/share/perl5/vendor_perl/Net/Amazon/S3/Client/Bucket.pm line 94
    Net::Amazon::S3::Client::Bucket::__ANON__('Data::Stream::Bulk::Callback=HASH(0x4427ad8)') called at /usr/share/perl5/vendor_perl/Data/Stream/Bulk/Callback.pm line 25
    Data::Stream::Bulk::Callback::get_more('Data::Stream::Bulk::Callback=HASH(0x4427ad8)') called at /usr/share/perl5/vendor_perl/Data/Stream/Bulk/DoneFlag.pm line 37
    Data::Stream::Bulk::DoneFlag::next('Data::Stream::Bulk::Callback=HASH(0x4427ad8)') called at /usr/share/perl5/vendor_perl/Data/Stream/Bulk.pm line 18
    Data::Stream::Bulk::items('Data::Stream::Bulk::Callback=HASH(0x4427ad8)') called at /check_pgbackrest/check_pgbackrest_tmp line 616
    main::check_wal_archives('HASH(0x1ae3c00)') called at /check_pgbackrest/check_pgbackrest_tmp line 816

I think it's related to self-signed certificates and I tried adding secure => 0 to the Net::Amazon::S3->new method without any luck. We'll have to find a way to take care of the repo1-s3-verify-ssl option if we wan't to accept this PR.

Kind regards

[pgstef]

Ok, I've been able to make it work with MinIO in http with secure => 0. I'll merge this PR and improve a bit the code (use /etc/pgbackrest.conf by default,...).

Thank you for your proposal.

Kind regards

[aebruno]

All sounds great. Makes sense to have this part of pgBackRest itself and having this merged will help out in the short term until this is fully integrated with pgBackRest. Thanks again!

[pgstef]

Just for the record, I've modified a little bit and pushed : https://github.com/dalibo/check_pgbackrest/commit/c4d163e00d18ba5dcdd75eb563f7f4e1a334af2e

It works with MinIO (HTTP). Would be great if you can confirm it works with a real S3 bucket.

Kind regards

[dwsteele]

BTW, check out the (currently undocumented) ls command: pgbackrest help ls that shipped with 2.15. It's a bit primitive at the moment, but with some improvements may allow you to innovate without input from the mother ship.

[pgstef]

@dwsteele indeed that ls command seems very promising especially if we could make it recursive. But then check_pgbackrest would require a specific pgbackrest version to work. Why not. I could follow pgbr release number to match compatibly. I'll keep an eye on that command for the next releases. Thanks for the tip !