search

arenadata / gpdb

Arenadata DB
https://docs.arenadata.io/en/ADB/current/introduction/intro.html
Apache License 2.0
40 stars 22 forks source link

Add tap-test for WAL files in archive_mode `always` #1067

Closed whitehawk closed 2 weeks ago

whitehawk commented 3 weeks ago

The commit adds tap-test to check that WAL files are not deleted at checkpoint.

Far ago, we've backported (https://github.com/arenadata/gpdb/commit/fa759d1bf70c020f884f05cf7ae15c6b9b47b4b1) archive_mode always to 6x. Since that, it was tested only manually. Later, there was a bug around WAL recycling depends on archive_mode. Upstream backported a fix (https://github.com/arenadata/gpdb/commit/4a5a95b3a62dadd73af73b76f8fed9deeea842a6), but the condition for XLogArchivingAlways() was not backported, because upstream doesn't support it. As a result, WAL files on mirrors may be considered as removable and can be actually removed during restart point. Backporting of https://github.com/arenadata/gpdb/commit/78ea8b5daab9237fd42d7a8a836c1c451765499f solves the problem, but new tap-test was also added to prove it works and to show such cases.

The upstream backport already covers WAL files on master instance, but I decided to add this check to test file too. Test file is heavily inspired by other test files from recovery dir.

(cherry picked from commit https://github.com/arenadata/gpdb/commit/65478088ae5d1fb9a5c8213cf9655d6d48c45b43)

Changes comparing to the original commit are:

  1. equivalent renamings due to porting from 6x to 7x;
  2. second checkpoint is removed as PostgreSQL version is > 11;
  3. pg_current_wal_insert_lsn() is used instead of pg_current_wal_lsn() when defining $walfile_to_be_kept value

Commits in the PR shouldn't be squashed to preserve authorship.

andr-sokolov commented 3 weeks ago

The task is to adopt patch from the related issue to our gpdb 7 fork. The patch is https://github.com/arenadata/gpdb/pull/877 Why is src/backend/access/transam/xlogarchive.c not changed in this PR?

whitehawk commented 3 weeks ago

The task is to adopt patch from the related issue to our gpdb 7 fork. The patch is #877 Why is src/backend/access/transam/xlogarchive.c not changed in this PR?

On GPDB 7x the problem is not reproduced, so changes in xlogarchive.c are not required. But it is usefull to port the test from the related issue. So this PR is only about the test.

andr-sokolov commented 3 weeks ago

Add to the description something about why CHECKPOINT has been removed from the original patch