search

uyuni-project / uyuni

Source code for Uyuni
https://www.uyuni-project.org/
GNU General Public License v2.0
434 stars 181 forks source link

Post-Upgrade Issues: Correcting VM Repository Configurations After Uyuni 2024.03 Upgrade #8832

Open Teamspeak5 opened 5 months ago

Teamspeak5 commented 5 months ago

Problem description

I recently upgraded 2022.10 to 2024.03 following the uyuni_upgrade_guide.pdf from 2022. After the upgrade, I saw multiple VMs from my assets failing the patching activities. While looking in to search more information about the issues with them, I noticed that all those VMs were not pointing at the Uyuni server but instead at 127.0.0.1

Most if not every of the VMs point to https://127.0.0.1:443/path_to_repo instead at it's FQDN after the upgrade. Even re-putting the repos from the vendor and re-take the subscription to the Uyuni channels helps solve the issue, this setting is imposed by Uyuni.

I tried finding configurations for the VMs when subscribing to a channel but found nothing about that. Most importantly, it's not right having this kind of big and impactful issues after a Major upgrade or any kind of upgrade and I would like to address this issue ASAP if possible.

Any kind of help to remediate the issue would be more than grateful! Please keep me posted, I'll upload any useful evidence to help further troubleshoot and solve the issue.

Steps to reproduce

  1. Have Uyuni with version 2022.10
  2. Login as root
  3. Run zypper ref
  4. Run zypper up susemanager
  5. Run the /usr/lib/susemanager/bin/server-migrator.sh script to upgrade the base OS and Uyuni Server.
  6. Run the the migrate script to migrate the database to the latest database version:/usr/lib/susemanager/bin/pg-migrate-x-to-y.sh
  7. reboot

Uyuni version

Loading repository data...
Reading installed packages...

Information for package Uyuni-Server-release:
---------------------------------------------
Repository     : Uyuni Server Stable
Name           : Uyuni-Server-release
Version        : 2024.05-230900.217.1.uyuni3
Arch           : x86_64
Vendor         : obs://build.opensuse.org/systemsmanagement:Uyuni
Support Level  : Level 3
Installed Size : 1.4 KiB
Installed      : Yes (automatically)
Status         : out-of-date (version 2024.03-230900.214.6.uyuni3 installed)
Source package : Uyuni-Server-release-2024.05-230900.217.1.uyuni3.src
Summary        : Uyuni Server
Description    :
    Uyuni lets you efficiently manage physical, virtual,
    and cloud-based Linux systems. It provides automated and cost-effective
    configuration and software management, asset management, and system
    provisioning.

Uyuni proxy version (if used)

N/A

Useful logs

N/A

Additional information

This server is part of a Hub XMLRPC API environment with 3 uyuni servers: (https://github.com/uyuni-project/hub-xmlrpc-api)

The issue seems to have appeared just on Uyuni01, I didn't take a good look at uyuni02 yet, but I'll do it if needed to compare results or do checks

rjmateus commented 5 months ago

Most importantly, it's not right having this kind of big and impactful issues after a Major upgrade or any kind of upgrade and I would like to address this issue ASAP if possible.

You are trying to upgrade from a version from 1 and a half years ago. Did you checked all release notes in between to see if any step was needed? Or to check if the upgrade was possible without an intermediate upgrade version?

You should check the config file at /etc/rhn/rhn.con and check variables cobbler.host (which should be localhost) and java.hostname (which should have your server DNS name).

If those variable are missing, add then with the values I mentioned.

Teamspeak5 commented 5 months ago

Most importantly, it's not right having this kind of big and impactful issues after a Major upgrade or any kind of upgrade and I would like to address this issue ASAP if possible.

You are trying to upgrade from a version from 1 and a half years ago. Did you checked all release notes in between to see if any step was needed? Or to check if the upgrade was possible without an intermediate upgrade version?

You should check the config file at /etc/rhn/rhn.con and check variables cobbler.host (which should be localhost) and java.hostname (which should have your server DNS name).

If those variable are missing, add then with the values I mentioned.

Hi @rjmateus, thank you for your feedback. Unfortunately no, I did not check all the release notes in between to check for further steps or intermediate upgrades, could you please provide me an example for those cases in order to pay more attention in the future and, hopefully, not encounter situations like that again?

I've checked the suggested configuration, cobbler.host was set to its FQDN and java.hostname was completely missing. I set the config as you suggested, now after re-subscribing the VM it's no longer setting the channels to localhost when subscribing a VM.

Could you please help me find and, possibly, address other issues and inconsistencies with uyuni01? Where should I better look to check for it's health and see if everything is as it should after this upgrade? Anyway, thank you again so much for the help

rjmateus commented 5 months ago

Could you please help me find and, possibly, address other issues and inconsistencies with uyuni01? You can look for errors in the logs, and if any feature is miss behaving. As far as I remember, if the server is running and you are able to see the minion and interact with them, then the migration should have worked fine.

From the release notes, the main thing to look at is when we ask to run the Major Upgrade procedure. That is when we normally do major changes, and if one skips it, it can cause problems on the server. Minor versions are fine to skip.

Teamspeak5 commented 5 months ago

Thank you, @rjmateus, I understood that. Additionally, as mentioned above :

Could you please help me find and, possibly, address other issues and inconsistencies with uyuni01? Where should I better look to check for it's health and see if everything is as it should after this upgrade? Anyway, thank you again so much for the help

Could you please help me find a way to better find for other possible problems/issues that occurred after the major upgrade to address them? Is there any way to achieve that, or I should just look at each release note and search for major changes?

I look forward to your feedback to attempt verification of possible extra “damage” to the server configurations/health