search

pulp / pulpcore

Pulp 3 pulpcore package https://pypi.org/project/pulpcore/
GNU General Public License v2.0
301 stars 116 forks source link

Delay publication file deletion to orphan cleanup to aid in online backups #2009

Open pulpbot opened 2 years ago

pulpbot commented 2 years ago

Author: @jlsherrill (jsherril@redhat.com)

Redmine Issue: 8852, https://pulp.plan.io/issues/8852

It would be useful if we could:

1) perform an online backup of the pulp postgresql database, this is relatively quick 2) backup the /var/lib/pulp/ filesystem

This is nearly possible today as we could simply not run orphan cleanup. However publication files (such as repomd files for yum content), are cleaned up when a publication is deleted. It would be awesome to defer that until orphan cleanup time. This would allow /var/lib/pulp/ to be additive until orphan cleanup is called.

pulpbot commented 2 years ago

From: @bmbouter (bmbouter) Date: 2021-06-02T18:58:26Z

I'd like to see this improve so +1 to this effort.

I have these concerns about the proposal (although I like the idea sharing):

  1. Other users may rely on this functionality and we probably couldn't adjust this with Pulp3 being released for so long.
  2. How can we find a solution that also serves users who want online backups but can't "hold back" the orphan cleanups?

Can more info be shared about the problems with an LVM snapshot approach? I've always heard that's the answer, but maybe in practice it's not that simple.

pulpbot commented 2 years ago

From: @jlsherrill (jsherril@redhat.com) Date: 2021-06-08T19:00:30Z

Evgeni was the one that shared the issues, essentially what it sounded like was that once you have a snapshot created, write performance drops considerably.

I found a blog post that details this: https://www.percona.com/blog/2009/02/05/disaster-lvm-performance-in-snapshot-mode/

stale[bot] commented 2 years ago

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!