gabjef
commented
1 week ago Please let me what else you might need to help diagnose this issue. I can provide the entire migration log if needed, but it's 1 million+ lines.
Open gabjef opened 1 week ago
gabjef
commented
1 week ago Please let me what else you might need to help diagnose this issue. I can provide the entire migration log if needed, but it's 1 million+ lines.
rjmateus
commented
1 week ago tomcat service looks to have failled to start. You should enter in the container terminal (mgrctl term) and then check the tomcat logs in the same way it was on the rpm based version (/var/log/tomcat/) and also the service logs.
gabjef
commented
1 week ago @rjmateus Thanks for the follow up. Yes we figured out that Tomcat is not starting.
The issue is that JAVA_OPTS, found in both /etc/tomcat/conf.d/remote_debug.conf and /etc/tomcat/tomcat.conf, is using options not supported by the JDK version used in the container:
uyuni-server:~ # systemctl status tomcat.service
× tomcat.service - Apache Tomcat Web Application Container
Loaded: loaded (/usr/lib/systemd/system/tomcat.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/tomcat.service.d
└─jmx.conf, override.conf
Active: failed (Result: exit-code) since Thu 2024-06-20 23:32:59 EDT; 3h 36min ago
Process: 810 ExecStart=/usr/lib/tomcat/server start (code=exited, status=1/FAILURE)
Main PID: 810 (code=exited, status=1/FAILURE)
Jun 20 23:32:59 uyuni-server.mgr.internal server[810]: classpath used: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/lib64/java/commons->
Jun 20 23:32:59 uyuni-server.mgr.internal server[810]: main class used: org.apache.catalina.startup.Bootstrap
Jun 20 23:32:59 uyuni-server.mgr.internal server[810]: flags used: -ea -Xms256m -Xmx1G -Djava.awt.headless=true -Dorg.xml.sax.driver=com.redhat.rhn.frontend.xmlrpc.util.>
Jun 20 23:32:59 uyuni-server.mgr.internal server[810]: options used: -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.>
Jun 20 23:32:59 uyuni-server.mgr.internal server[810]: arguments used: start
Jun 20 23:32:59 uyuni-server.mgr.internal server[810]: Unrecognized VM option 'UseConcMarkSweepGC'
Jun 20 23:32:59 uyuni-server.mgr.internal server[810]: Error: Could not create the Java Virtual Machine.
Jun 20 23:32:59 uyuni-server.mgr.internal server[810]: Error: A fatal exception has occurred. Program will exit.
Jun 20 23:32:59 uyuni-server.mgr.internal systemd[1]: tomcat.service: Main process exited, code=exited, status=1/FAILURE
Jun 20 23:32:59 uyuni-server.mgr.internal systemd[1]: tomcat.service: Failed with result 'exit-code'.UseConcMarkSweepGC for garbage collection is not supported in JDK version 17 (17.0.11 in the container). Note that the JDK version on the source Uyuni server is 11.0.23.
Actually, we had to remove 2 additional options for Tomcat to finally start successfully:
MaxNewSize=256--add-modules java.annotation,com.sun.xml.bindHere is the 'default' JAVA_OPTS causing Tomcat to fail:
JAVA_OPTS="-ea -Xms256m -Xmx1G -Djava.awt.headless=true -Dorg.xml.sax.driver=com.redhat.rhn.frontend.xmlrpc.util.RhnSAXParser -Dorg.apache.tomcat.util.http.Parameters.MAX_COUNT=1024 -XX:MaxNewSize=256 -XX:-UseConcMarkSweepGC -Dnet.sf.ehcache.skipUpdateCheck=true --add-exports java.annotation/javax.annotation.security=ALL-UNNAMED --add-opens java.annotation/javax.annotation.security=ALL-UNNAMED --add-modules java.annotation,com.sun.xml.bind"This trimmed down JAVA_OPTS worked for us:
JAVA_OPTS="-ea -Xms256m -Xmx1G -Djava.awt.headless=true -Dorg.xml.sax.driver=com.redhat.rhn.frontend.xmlrpc.util.RhnSAXParser -Dorg.apache.tomcat.util.http.Parameters.MAX_COUNT=1024 -Dnet.sf.ehcache.skipUpdateCheck=true --add-exports java.annotation/javax.annotation.security=ALL-UNNAMED --add-opens java.annotation/javax.annotation.security=ALL-UNNAMED"So we were able to get Spacewalk/Uyuni functionality up and running using the trimmed down version of JAVA_OPTS.
Note that the following issues still exist:
uyuni-server-attestation.service is still not found by mgradm statusmgradm inspect is still failing
Problem description
After migration, from "legacy" 2024.05 Uyuni server to Podman container on openSUSE Leap Micro, Uyuni/Spacewalk is not functional on new containerized
uyuni-serverinstanceuyuni-serveruyuni-server-attestation.serviceis not found bymgradm statusmgradm inspectfailspodman psshowsuyuni-serverasunhealthyService unavailable!)Steps to reproduce
mgradm migrate podman uyuni-lab-linux-mgmt.lab.tierpoint.comUyuni version
--- System Info & Versions ---
Source Legacy System / openSUSE Leap 15.5 - uyuni-lab-linux-mgmt.lab.tierpoint.com
Target Container System / openSUSE Leap Micro 5.5 - uyuni-lab-micro-suse-migration.lab.tierpoint.com
Uyuni proxy version (if used)
Useful logs
--- Perform migration ---
--- Checking status with
mgradm status---First note the failed status check for
uyuni-server-attestation.serviceRest of the services seem to be there
---
mgradm inspectfails ------
podman psshows container as unhealthy ------ Shell into the container and check for Spacewalk & port 8009 there ---
Try
spacecmdfirstNothing on port 8009
--- Database status (seems okay) ---
Additional information
TBD