New Uyuni install fails

darrynl commented 2 months ago

Problem description

When doing a fresh installation of Uyuni, the Uyuni setup fails.

Steps to reproduce

Build new openSUSE 15.5 VM from openSUSE-Leap-15.5-DVD-x86_64-Build491.1-Media.iso. 4 cores, 16GB RAM, 40GB root filesytem, 5GB var filesystem, 10BG /var/cache filesystem, 3 GB swap
After build is complete, add 102GB /var/spacewalk filesystem and 52GB /var/lib/pgsql filesystem
Use zypper to confirm all updates applied.
Add entry to /etc/hosts so that machine can resolve it's own FQDN.
Install patterns-uyuni_server exactly as per: https://www.uyuni-project.org/uyuni-docs/en/uyuni/installation-and-upgrade/install-uyuni.html
Run Uyuni setup as detailed: https://www.uyuni-project.org/uyuni-docs/en/uyuni/installation-and-upgrade/uyuni-server-setup.html I made sure the passwords used adhere to the requirements specificed.
Setup fails with "Could not restart spacewalk services. Exit value: 1."

...

Uyuni version

Information for package Uyuni-Server-release:
---------------------------------------------
Repository     : uyuni-server-stable
Name           : Uyuni-Server-release
Version        : 2024.05-230900.217.1.uyuni3
Arch           : x86_64
Vendor         : obs://build.opensuse.org/systemsmanagement:Uyuni
Support Level  : Level 3
Installed Size : 1.4 KiB
Installed      : Yes (automatically)
Status         : up-to-date
Source package : Uyuni-Server-release-2024.05-230900.217.1.uyuni3.src
Summary        : Uyuni Server
Description    :
    Uyuni lets you efficiently manage physical, virtual,
    and cloud-based Linux systems. It provides automated and cost-effective
    configuration and software management, asset management, and system
    provisioning.

Uyuni proxy version (if used)

N/A

Useful logs

Installation log (domain name replaced with xxxxxx):
Installation log of Uyuni
Command: /usr/bin/spacewalk-setup

hibernate.connection.driver_proto
hibernate.dialect
hibernate.connection.driver_class
readline() on unopened filehandle TARGET at /usr/bin/rhn-config-satellite.pl line 72.
hibernate.connection.driver_proto
hibernate.connection.driver_class
hibernate.dialect
db_host
db_name
db_password
db_user
db_port
db_backend
db_host
db_backend
db_name
db_password
db_port
db_user
Shutting down spacewalk services...
Done.
Enabling spacewalk services...
Done
product_name
/var/lib/rhn/rhn-satellite-prep/etc/rhn/rhn.conf
/var/lib/rhn/rhn-satellite-prep/etc/rhn/rhn.conf.2024-07-03_09:53:25.067683
/var/lib/rhn/rhn-satellite-prep/etc -> /etc
/var/lib/rhn/rhn-satellite-prep/etc/rhn -> /etc/rhn
/var/lib/rhn/rhn-satellite-prep/etc/rhn/rhn.conf -> /etc/rhn/rhn.conf
 * Making backup of /etc/rhn/rhn.conf to /etc/sysconfig/rhn/backup-2024-07-03-11:53/etc/rhn/rhn.conf
/var/lib/rhn/rhn-satellite-prep/etc/rhn/rhn.conf.2024-07-03_09:53:25.067683 -> /etc/rhn/rhn.conf.2024-07-03_09:53:25.067683
Generating private CA key: /root/ssl-build/RHN-ORG-PRIVATE-SSL-KEY

Generating public CA certificate: /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT
Using distinguishing variables:
    --set-country      = "ZA"
    --set-state        = "Western Cape"
    --set-city         = "Cape Town"
    --set-org          = "xxxxxx"
    --set-org-unit     = "Technical"
    --set-common-name  = "apoc.xxxxxx.co.za"
    --set-email        = ""
CA Cert for OS Images: Packaging /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT into /srv/susemanager/salt/images/rhn-org-trusted-ssl-cert-osimage-1.0-1.noarch.rpm
CA Cert for OS Images: Packaging /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT into /srv/susemanager/salt/images/rhn-org-trusted-ssl-cert-osimage-sle11-1.0-1.noarch.rpm

Generating the web server's SSL private key: /root/ssl-build/apoc.xxxxxx/server.key

Generating web server's SSL certificate request: /root/ssl-build/apoc.xxxxxx/server.csr
Using distinguished names:
    --set-country      = "ZA"
    --set-state        = "Western Cape"
    --set-city         = "Cape Town"
    --set-org          = "xxxxxx"
    --set-org-unit     = "Technical"
    --set-hostname     = "apoc.xxxxxx.co.za"
    --set-email        = "uyuni@xxxxxx.co.za"
Rotated: rhn-ca-openssl.cnf --> rhn-ca-openssl.cnf.1

Generating/signing web server's SSL certificate: server.crt
After changing the server certificate please execute:
$> spacewalk-service stop
$> systemctl restart postgresql.service
$> spacewalk-service start

As the CA certificate has been changed, please deploy the CA to all registered clients.
On salt-managed clients, you can do this by applying the highstate.
/var/lib/rhn/rhn-satellite-prep/etc/rhn -> /etc/rhn
/var/lib/rhn/rhn-satellite-prep/etc/rhn/rhn.conf -> /etc/rhn/rhn.conf
/var/lib/rhn/rhn-satellite-prep/etc/rhn/rhn.conf.2024-07-03_09:53:25.067683 -> /etc/rhn/rhn.conf.2024-07-03_09:53:25.067683
 * Making backup of /etc/rhn/rhn.conf.2024-07-03_09:53:25.067683 to /etc/sysconfig/rhn/backup-2024-07-03-11:53/etc/rhn/rhn.conf.2024-07-03_09:53:25.067683
Shutting down spacewalk services...
Done.
Starting spacewalk services...
  Checking DB schema and running DB schema upgrade if needed. This may take a while.
  Call the following command to see progress: journalctl -f -u uyuni-check-database.service
A dependency job for spacewalk.target failed. See 'journalctl -xe' for details.
FAILED

journalctl -xe
apoc:/var/log/rhn # journalctl -xe
Jul 03 11:54:14 apoc server[4406]: 03-Jul-2024 11:54:14.625 INFO [pool-4-thread-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illega>
Jul 03 11:54:14 apoc server[4406]:         java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [jav>
Jul 03 11:54:14 apoc server[4406]:                 at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1349)
Jul 03 11:54:14 apoc server[4406]:                 at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1337)
Jul 03 11:54:14 apoc server[4406]:                 at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1174)
Jul 03 11:54:14 apoc server[4406]:                 at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1141)
Jul 03 11:54:14 apoc server[4406]:                 at com.suse.manager.webui.websocket.WebsocketHeartbeatService.clearStaleSessions(WebsocketHeartbeatService.java:84)
Jul 03 11:54:14 apoc server[4406]:                 at com.suse.manager.webui.websocket.WebsocketHeartbeatService.lambda$initScheduler$2(WebsocketHeartbeatService.java:>
Jul 03 11:54:14 apoc server[4406]:                 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
Jul 03 11:54:14 apoc server[4406]:                 at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
Jul 03 11:54:14 apoc server[4406]:                 at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.ja>
Jul 03 11:54:14 apoc server[4406]:                 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Jul 03 11:54:14 apoc server[4406]:                 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Jul 03 11:54:14 apoc server[4406]:                 at java.base/java.lang.Thread.run(Thread.java:829)
Jul 03 11:54:30 apoc sshd[6726]: Accepted keyboard-interactive/pam for root from 192.168.101.115 port 55227 ssh2
Jul 03 11:54:30 apoc systemd-logind[971]: New session 3 of user root.
░░ Subject: A new session 3 has been created for user root
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ Documentation: sd-login(3)
░░
░░ A new session with the ID 3 has been created for the user root.
░░
░░ The leading process of the session is 6726.
Jul 03 11:54:30 apoc systemd[1]: Started Session 3 of User root.
░░ Subject: A start job for unit session-3.scope has finished successfully
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit session-3.scope has finished successfully.
░░
░░ The job identifier is 2593.
Jul 03 11:54:30 apoc sshd[6726]: pam_unix(sshd:session): session opened for user root by (uid=0)
Jul 03 11:55:17 apoc spacewalk-startup-helper[6094]: taskomatic still not up and running
Jul 03 11:55:17 apoc systemd[1]: Finished Spacewalk wait for taskomatic.
░░ Subject: A start job for unit spacewalk-wait-for-taskomatic.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit spacewalk-wait-for-taskomatic.service has finished successfully.
░░
░░ The job identifier is 1843.
Jul 03 11:55:20 apoc sshd[6904]: Connection closed by 192.168.101.15 port 34582 [preauth]

Additional information

No response

avshiliaev commented 2 months ago

Hi @darrynl,

Thank you for the report. To diagnose the issue further, please follow these steps:

Check the status of spacewalk.target for any dependency failures:
```
systemctl status spacewalk.target
```
Review the detailed logs for uyuni-check-database.service:
```
journalctl -f -u uyuni-check-database.service
```
Ensure PostgreSQL is running correctly:
```
systemctl status postgresql.service
```

Please provide the output of these commands for further analysis.

bmswork commented 1 month ago

@avshiliaev I've got the same issue, taskomatic appears to be the issue. Postgres is fine, spacewalk is down on dependencies.

2024-07-09 14:58:55,627 [Thread-41] FATAL com.redhat.rhn.taskomatic.core.TaskomaticDaemon - java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy$SuffixingRandom$BaseNameResolver
        at org.hibernate.cfg.Environment.buildBytecodeProvider(Environment.java:357)
        at org.hibernate.cfg.Environment.buildBytecodeProvider(Environment.java:352)
        at org.hibernate.cfg.Environment.<clinit>(Environment.java:246)
        at org.hibernate.boot.registry.StandardServiceRegistryBuilder.<init>(StandardServiceRegistryBuilder.java:122)
        at org.hibernate.boot.registry.StandardServiceRegistryBuilder.<init>(StandardServiceRegistryBuilder.java:96)
        at org.hibernate.cfg.Configuration.reset(Configuration.java:158)
        at org.hibernate.cfg.Configuration.<init>(Configuration.java:124)
        at org.hibernate.cfg.Configuration.<init>(Configuration.java:118)
        at com.redhat.rhn.common.hibernate.AbstractConnectionManager.createSessionFactory(AbstractConnectionManager.java:182)
        at com.redhat.rhn.common.hibernate.AbstractConnectionManager.initialize(AbstractConnectionManager.java:166)
        at com.redhat.rhn.common.hibernate.HibernateFactory.createSessionFactory(HibernateFactory.java:119)
        at com.redhat.rhn.taskomatic.core.SchedulerKernel.startup(SchedulerKernel.java:131)
        at com.redhat.rhn.taskomatic.core.TaskomaticDaemon.lambda$onStartup$0(TaskomaticDaemon.java:154)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassNotFoundException: net.bytebuddy.NamingStrategy$SuffixingRandom$BaseNameResolver
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
        ... 14 more

mcalmer commented 1 month ago

@bmswork Please downgrade "bytebuddy" package. Leap released an incompatible version. We will fix this in a future release

bmswork commented 1 month ago

Wonderful, that worked just fine: zypper -n install --auto-agree-with-licenses --oldpackage 'byte-buddy < 1.12'

EthanB11 commented 1 month ago

@mcalmer is there an estimated release date for the newer package as the current one is old and have open vulnerabilities. https://nvd.nist.gov/vuln/detail/CVE-2021-33813

mcalmer commented 1 month ago

AFAIK all changes were checked-in yesterday so Uyuni 2024.08 will have it.

mcalmer commented 1 month ago

The link you send was about jdom and not byte-buddy. AFAIK that CVE is fixed since a while already

uyuni-project / uyuni