search

Katello / katello-packaging

[DEPRECATED] Packaging for Katello
7 stars 33 forks source link

Fixes #20550 - support for remote DB in backup and restore #499

Closed mbacovsky closed 6 years ago

mbacovsky commented 7 years ago

Backup and restore scripts were changed to access postgress DB using the configuration from Foreman and Candlepin and not directly as DB superuser.

In online backup we skip dump of global objects as usually there are no privileges to restore them on the remote DB.

In offline backup we do ofline backup as before only the remote DBs are backed up using pg_dump as in online backup because we do not have direct access to /var/lib/pgsql/data. We assume the remote DB is always up.

The restore was updated accordingly and can restore offline backup including dumps of the remote dbs.

katello-service was fixed to not touch postgresql service when the DB is remote.

theforeman-bot commented 7 years ago

@mbacovsky, the Redmine ticket used is for a different project than the one associated with this GitHub repository. Please either:

If changing the ticket number used, remember to update the PR title and the commit message (using git commit --amend).

This message was auto-generated by Foreman's prprocessor

theforeman-bot commented 7 years ago

Issues: #20550

mbacovsky commented 7 years ago

The installer --reset is dropping the DBs and the owner usually do not have the permission to re-create them. I'll add small fix skipping the drop db for remote dbs.

Other then that I was able to backup and restore Katrello completely on remote DB.

It would be great if someone can check that the changes I had to do to the backup procedure make sense and are okay with what is expected by admins in the field.

mbacovsky commented 7 years ago

Unfortunately I'm not going to finish my updates before I leave for vacation. I'll update 1st week of September.

theforeman-bot commented 7 years ago

@mbacovsky, the Redmine ticket used is for a different project than the one associated with this GitHub repository. Please either:

If changing the ticket number used, remember to update the PR title and the commit message (using git commit --amend).

This message was auto-generated by Foreman's prprocessor

mbacovsky commented 7 years ago

@sean797, @beav , I've updated this PR. I've extracted the common code and put it to the /usr/share/foreman/script. Feel free to suggest better location. I've also addressed the comments above and rebased the PR. The katello-restore requires change in how the --reset feature of the installer approaches the db. The PR will follow soon.

sean797 commented 7 years ago

@mbacovsky I would prefer if it could go in lib/ like https://github.com/Katello/katello-packaging/pull/524/

Overall looks good 👍 but I haven't tested it.

mbacovsky commented 6 years ago

@sean797 I'll wait till the other PR is merged and update the code accordingly. lib/ is more convenient place for the db handling code.

theforeman-bot commented 6 years ago

@mbacovsky, the Redmine ticket used is for a different project than the one associated with this GitHub repository. Please either:

If changing the ticket number used, remember to update the PR title and the commit message (using git commit --amend).

This message was auto-generated by Foreman's prprocessor

theforeman-bot commented 6 years ago

@mbacovsky, the Redmine ticket used is for a different project than the one associated with this GitHub repository. Please either:

If changing the ticket number used, remember to update the PR title and the commit message (using git commit --amend).

This message was auto-generated by Foreman's prprocessor

mbacovsky commented 6 years ago

@sean797 I've updated the patch to work with the refactored scripts. I've updated the SPEC and tested the scratch builds with rest of the packages from Katello nightly. I was able to backup and restore my Katello successfully. There were some hiccups during db:migrate and internal proxy check. It was fixed after re-running the installer. I'm not sure if that is a nightly thing or my semi-devel box upgraded from Katello 3.4.

There is one fix in progress to prevent dropping remote DB in reset hook. I'll send it later today.

mbacovsky commented 6 years ago

@sean797, refactored the pg_command a bit...

mbacovsky commented 6 years ago

@sean797 thanks for all the reviewing effort!

ehelms commented 6 years ago

@cfouant can you give this a once over?

mbacovsky commented 6 years ago

@ekohl there are plans to move backup and restore to the foreman-maintain and I believe it is a matter of the resources now. Is it @ehelms?

In foreman-maintain some time ago I started the work on extending the features representing foreman and candlepin databases to support remote DBs. I used the service configs to obtain the db connection data so it is not necessary to couple it that tightly with puppet modules (answer files). Not sure which approach is better.

ehelms commented 6 years ago

That is the general plan, although no one currently owns migrating them to foreman-maintain. But we have talked about it, and proposed it via an RFC. I don't think it would be a ton of work, more work on the testing side, and updating docs to incorporate using foreman-maintain to perform the actions versus these scripts.

ehelms commented 6 years ago

These packages and code have all moved to foreman-packaging rpm/develop under a katello subfolder. Please re-open your PR against it to ensure there is no data loss between the repositories. The code in this repository will be subsequently removed -- https://github.com/Katello/katello-packaging/pull/575

johnpmitsch commented 6 years ago

This functionality is being added to foreman maintain and here: https://github.com/Katello/katello-installer/pull/550