search

ossc-db / pg_rman

Backup and restore management tool for PostgreSQL
http://ossc-db.github.io/pg_rman/index.html
Other
476 stars 77 forks source link

How to remove old backup's archive log #194

Closed 8ugMak1r closed 3 years ago

8ugMak1r commented 3 years ago

The full backup and increment backup also with archive backup, and the archive logs take lot of space. Is it possible to only backup data when full backup or increment backup and restore at the backup time?

mikecaat commented 3 years ago

Hmm, my understanding is that there is no feature you desired now and I'm not sure how it's useful for users. IMHO, if the option you desired is exists, some users worry that "I can't restore the database cluster to the specific point in time although I took backup with pg_rman..."

BTW, do the options "--keep-arclog-files / --keep-arclog-days" enough for your use-case? If you remove the archive logs after backup is taken with those options, I think you can reduce the space.

8ugMak1r commented 3 years ago

Thank you for your reply @mikecaat Sometimes, I don't need the PITR feature, so the archive logs are not needed, and I only want increment backup of the DATA. Options "--keep-arclog-files / --keep-arclog-days" seem like only save the space of the origin server, I wanna save the backup space too.

mikecaat commented 3 years ago

Thanks. OK, I see there is a use-case for the option which disables archiving WAL. I'm not sure when we can add the feature because we don't have enough development resource.

MoonInsung commented 3 years ago

Dear @hard88

Thank you for suggestions on new features.

Like your problem, if lot of archive logs on the server, huge size of backup files can be created each time perform a full backup/incremental backup. Therefore, to solve this problem, we recommend using --keep-arclog--xxxx, an option to delete unnecessary archive logs prior to the point of backup.

The reason we recommend using --keep-arclog--xxxx option, is that if there are a lot of backup files from A to Z, we can restore them at any time in between A to Z. In other words, it is because I think it is good to be able to restore at any point while minimizing the backup file size(using --keep-arclog-xxxx options).

Because, if there is a backup file without an archive log at any one of backup files (include arc log backup -> include arc log backup -> exclude arc log backup -> include arc log backup...), a point in time at which restore is impossible occurs(PITR). Therefore, if exclude for the archive log due to a user's mistake when executing the backup, it may not be possible to restore to the desired point in time, and the worst accident may occur.

So, that's why I think it's risky to add an option that causes these problems.

Best regards. Moon.

8ugMak1r commented 3 years ago

Thank you for your reply. @MoonInsung @mikecaat Please forgive me that I didn't make myself clear. I should change the title to "How to remove old backup's archive log" Suppose I have three backups: A(full) on January 1st, B(increment) on February 1st, C(increment) on March 1st, by PG_RMAN, I can easily make a PITR between A and C, that great. When April 1st, I make an increment backup D, so I can make PITR between A and D, but I don't need PITR between A and B anymore, the ability to make a PITR between B and D is enough for me. So If I delete the archive logs between A and B, it still works well for me and also free my backup device's space.

mikecaat commented 3 years ago

Thanks, I understood there is happy case clearly. But, I think this feature should not be implemented although this is just my thought. Sorry... I also agree @MoonInsung 's comment that it's too dangerous for some users. They may mistake without knowing what a dangerous operation they are doing.

8ugMak1r commented 3 years ago

I manually deleted the old backup's archive log and made a PITR, got a warning message but PG_RMAN still worked. I will take more tests for this. Thank you for your time.