Closed astylianGR closed 2 years ago
This error only happens when creating an AUX image with Podman and OL 8 If the WKT UI is run with root access there is no error.
log with more info:
java.io.IOException: Build command failed with error: Error: kernel does not support overlay fs: unable to create kernel-style whiteout: operation not permitted
at com.oracle.weblogic.imagetool.util.Utils.processError(Utils.java:272)
at com.oracle.weblogic.imagetool.builder.BuildCommand.run(BuildCommand.java:155)
at com.oracle.weblogic.imagetool.cli.menu.CommonOptions.runDockerCommand(CommonOptions.java:107)
at com.oracle.weblogic.imagetool.cli.menu.CreateAuxImage.call(CreateAuxImage.java:50)
at com.oracle.weblogic.imagetool.cli.menu.CreateAuxImage.call(CreateAuxImage.java:21)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at com.oracle.weblogic.imagetool.cli.ImageTool.run(ImageTool.java:79)
at com.oracle.weblogic.imagetool.cli.ImageTool.main(ImageTool.java:51)
<2022.06.28 21:06:39>
<> <2022.06.28 21:06:39>
<> <ENTRY [/usr/bin/podman, image, prune, -f, --filter, label=com.oracle.weblogic.imagetool.buildid=0237e704-a5bb-4d55-bb65-3fb12457736a]> <2022.06.28 21:06:39>
<>
When the Image Tool using Podman was called from the command line in OL 8 it did not have any errors.
@astylianGR This appears to be a podman rootless operation related.
Can you describe your environment ?
Is this an Oracle Cloud compute instance?
Any NFS involved ?
Can you share the version of podman and the file contents in ~/.config/containers (if any)?
Also, have you tried to run it as a different non root user ? (i.e. not opc ) .
Have you run and dnf update?
I have been using the UI to create aux image using a new user I created in the oci ol8 compute instance without any issue.
Hi @jshum2479 this is the info you asked for:
OCI Image: Oracle-Linux-8.6-2022.05.31-0
[opc@vm-adm-prod ~]$ cat /etc/release Oracle Linux Server release 8.6 NAME="Oracle Linux Server" VERSION="8.6" ID="ol" ID_LIKE="fedora" VARIANT="Server" VARIANT_ID="server" VERSION_ID="8.6" PLATFORM_ID="platform:el8" PRETTY_NAME="Oracle Linux Server 8.6" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:oracle:linux:8:6:server" HOME_URL="https://linux.oracle.com/" BUG_REPORT_URL="https://bugzilla.oracle.com/"
ORACLE_BUGZILLA_PRODUCT="Oracle Linux 8" ORACLE_BUGZILLA_PRODUCT_VERSION=8.6 ORACLE_SUPPORT_PRODUCT="Oracle Linux" ORACLE_SUPPORT_PRODUCT_VERSION=8.6 Red Hat Enterprise Linux release 8.6 (Ootpa) Oracle Linux Server release 8.6 cpe:/o:oracle:linux:8:6:server [opc@vm-adm-prod ~]$
Info about pod man:
[root@vm-adm-prod ~]# rpm -qa | grep -i podman podman-catatonit-4.0.2-6.module+el8.6.0+20665+a3b29bef.x86_64 podman-docker-4.0.2-6.module+el8.6.0+20665+a3b29bef.noarch podman-4.0.2-6.module+el8.6.0+20665+a3b29bef.x86_64 cockpit-podman-43-1.module+el8.6.0+20665+a3b29bef.noarch [root@vm-adm-prod ~]#
[opc@vm-adm-prod ~]$ docker -v Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. podman version 4.0.2
[opc@vm-adm-prod ~]$ df Filesystem 1K-blocks Used Available Use% Mounted on devtmpfs 8004972 0 8004972 0% /dev tmpfs 8045036 21052 8023984 1% /dev/shm tmpfs 8045036 255032 7790004 4% /run tmpfs 8045036 0 8045036 0% /sys/fs/cgroup /dev/mapper/ocivolume-root 37177616 23125724 14051892 63% / /dev/sda2 1038336 542108 496228 53% /boot /dev/mapper/ocivolume-oled 10475520 170932 10304588 2% /var/oled /dev/sda1 102182 5060 97122 5% /boot/efi tmpfs 1609004 0 1609004 0% /run/user/0 tmpfs 1609004 96 1608908 1% /run/user/1000 tmpfs 1609004 4 1609000 1% /run/user/987 [opc@vm-adm-prod ~]$
Let me know if you require more info
Hmm, I have similar environment. I don't have docker installed in my environment.
cat /etc/os-release NAME="Oracle Linux Server" VERSION="8.6" ID="ol" ID_LIKE="fedora" VARIANT="Server" VARIANT_ID="server" VERSION_ID="8.6" PLATFORM_ID="platform:el8" PRETTY_NAME="Oracle Linux Server 8.6" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:oracle:linux:8:6:server" HOME_URL="https://linux.oracle.com/" BUG_REPORT_URL="https://bugzilla.oracle.com/"
ORACLE_BUGZILLA_PRODUCT="Oracle Linux 8" ORACLE_BUGZILLA_PRODUCT_VERSION=8.6 ORACLE_SUPPORT_PRODUCT="Oracle Linux" ORACLE_SUPPORT_PRODUCT_VERSION=8.6
[root@johnny-oel8 ~]# rpm -qa | grep -i podman podman-catatonit-4.0.2-6.module+el8.6.0+20665+a3b29bef.x86_64 podman-4.0.2-6.module+el8.6.0+20665+a3b29bef.x86_64
and it's working both opc or oracle (I added) user.
Did you install docker ?
Can you check this also?
rpm -qa | grep fuse fuse3-3.3.0-15.0.2.el8.x86_64 fuse3-libs-3.3.0-15.0.2.el8.x86_64 fuse-common-3.3.0-15.0.2.el8.x86_64 fuse-2.9.7-15.0.2.el8.x86_64 fuse-overlayfs-1.8.2-1.module+el8.6.0+20665+a3b29bef.x86_64 gvfs-fuse-1.36.2-11.el8.x86_64 fuse-libs-2.9.7-15.0.2.el8.x86_64
[opc@vm-adm-prod ~]$ rpm -qa | grep fuse fuse3-3.3.0-15.0.2.el8.x86_64 fuse-common-3.3.0-15.0.2.el8.x86_64 gvfs-fuse-1.36.2-11.el8.x86_64 fuse-libs-2.9.7-15.0.2.el8.x86_64 fuse3-libs-3.3.0-15.0.2.el8.x86_64 fuse-overlayfs-1.8.2-1.module+el8.6.0+20665+a3b29bef.x86_64 fuse-2.9.7-15.0.2.el8.x86_64
Yes I have installed docker initially... then I saw that I don't needed... I have configured podman in the UI.
I have removed the podman-docker-2:4.0.2-6.module+el8.6.0+20665+a3b29bef.noarch and opened UI as opc and the issue is still reproducible
JDK version is 11.0.15.1+2-LTS-10 JAVA_HOME = /usr/java/jdk-11.0.15.1 CLASSPATH = /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/lib/weblogic-deploy-core.jar:/u01/Middleware/wlserver/server/lib/weblogic.jar JAVA_PROPERTIES = -Djava.util.logging.config.class=oracle.weblogic.deploy.logging.WLSDeployLoggingConfig -Dpython.cachedir.skip=true -Dpython.console= /usr/java/jdk-11.0.15.1/bin/java -cp /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/lib/weblogic-deploy-core.jar:/u01/Middleware/wlserver/server/lib/weblogic.jar -Djava.util.logging.config.class=oracle.weblogic.deploy.logging.WLSDeployLoggingConfig -Dpython.cachedir.skip=true -Dpython.console= -Dpython.path=/u01/Middleware/wlserver/common/wlst/modules/jython-modules.jar/Lib org.python.util.jython /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/lib/python/prepare_model.py -oracle_home /u01/Middleware -model_file /home/opc/edea-oke-prod-models/model.yaml -output_dir /home/opc/prepareModelxEoetU -target wko
Issue Log for prepareModel version 2.1.0 running WebLogic version 14.1.1.0.0 offline mode:
Total: WARNING : 0 SEVERE : 0
prepareModel.sh completed successfully (exit code = 0) /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/bin/prepareModel.sh exited with exit code 0 [FINER ] ENTRY [INFO ] WebLogic Image Tool version 1.11.0 [INFO ] Image Tool build ID: 9e8b899d-11d4-4bb8-8bc4-9469703f9b5b [FINER ] Discovered proxy setting (http): null [FINER ] Discovered proxy setting (https): null [FINER ] Discovered proxy setting (none): null [FINER ] RETURN [INFO ] Temporary directory used for image build context: /tmp/wlsimgbuilder_temp9787583822912561244 [FINER ] ENTRY /tmp/wlsimgbuilder_temp9787583822912561244 [INFO ] Copying /home/opc/edea-oke-prod-models/model.yaml to build context folder. [FINER ] ENTRY wdt 2.3.1 [FINER ] RETURN [FINER ] ENTRY [FINER ] RETURN [FINER ] ENTRY [FINER ] ENTRY wdt_2.3.1 [FINER ] RETURN /home/opc/Downloads/weblogic-deploy.tar.gz [INFO ] Copying /home/opc/Downloads/weblogic-deploy.tar.gz to build context folder. [FINER ] RETURN /tmp/wlsimgbuilder_temp9787583822912561244/weblogic-deploy.tar.gz [FINER ] RETURN [FINER ] ENTRY [FINER ] RETURN [INFO ] Starting build: /usr/bin/podman build --no-cache --force-rm --tag lhr.ocir.io/lrtg4a4iixwu/cfl-prod-reg-aux-image:4 /tmp/wlsimgbuilder_temp9787583822912561244 [FINER ] ENTRY [/usr/bin/podman, build, --no-cache, --force-rm, --tag, lhr.ocir.io/lrtg4a4iixwu/cfl-prod-reg-aux-image:4, /tmp/wlsimgbuilder_temp9787583822912561244] null [FINER ] Docker log: null [FINER ] Starting docker process... [FINER ] Docker process started [FINER ] Waiting for Docker to finish [FINE ] ERROR java.io.IOException: Build command failed with error: Error: kernel does not support overlay fs: unable to create kernel-style whiteout: operation not permitted
at com.oracle.weblogic.imagetool.util.Utils.processError(Utils.java:272)
at com.oracle.weblogic.imagetool.builder.BuildCommand.run(BuildCommand.java:155)
at com.oracle.weblogic.imagetool.cli.menu.CommonOptions.runDockerCommand(CommonOptions.java:107)
at com.oracle.weblogic.imagetool.cli.menu.CreateAuxImage.call(CreateAuxImage.java:50)
at com.oracle.weblogic.imagetool.cli.menu.CreateAuxImage.call(CreateAuxImage.java:21)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at com.oracle.weblogic.imagetool.cli.ImageTool.run(ImageTool.java:79)
at com.oracle.weblogic.imagetool.cli.ImageTool.main(ImageTool.java:51)
[FINER ] ENTRY [FINER ] ENTRY [/usr/bin/podman, image, prune, -f, --filter, label=com.oracle.weblogic.imagetool.buildid=9e8b899d-11d4-4bb8-8bc4-9469703f9b5b] java.io.IOException: Build command failed with error: Error: kernel does not support overlay fs: unable to create kernel-style whiteout: operation not permitted
at com.oracle.weblogic.imagetool.util.Utils.processError(Utils.java:272)
at com.oracle.weblogic.imagetool.util.Utils.runDockerCommand(Utils.java:250)
at com.oracle.weblogic.imagetool.util.Utils.removeIntermediateDockerImages(Utils.java:706)
at com.oracle.weblogic.imagetool.cli.menu.CommonOptions.cleanup(CommonOptions.java:240)
at com.oracle.weblogic.imagetool.cli.menu.CreateAuxImage.call(CreateAuxImage.java:55)
at com.oracle.weblogic.imagetool.cli.menu.CreateAuxImage.call(CreateAuxImage.java:21)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at com.oracle.weblogic.imagetool.cli.ImageTool.run(ImageTool.java:79)
at com.oracle.weblogic.imagetool.cli.ImageTool.main(ImageTool.java:51)
[FINE ] User requested usage by using help command /opt/WebLogic Kubernetes Toolkit UI/tools/imagetool/bin/imagetool.sh exited with exit code 0
logs for the UI with logger to finer
How do you launch the UI ? From the desktop or through a terminal? I use the desktop, search for UI and then launch it. Can you modify the the imagetool.sh and echo the PATH to see what it is?
Mine is like this
PATH IS /home/opc/.local/bin:/home/opc/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbinundefined
I also use the Desktop and this is the path
/home/opc/.local/bin:/home/opc/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/binundefined
The undefined
appended to the end messed up the environment. podman is in /usr/bin. If you force /usr/bin in the front of the path, then I think it will work. We will try to track down where that undefined
comes from.
We found the problem, it should be fix in next release which may happen today.
great please let me know when the release will be ready for download and I will also notify the customer.
Thanks!
@astylianGR 1.2.0 has been released. Please try it out and let us know if this resolved the problem.
Closing due to lack of user feedback on the delivered fix.
When using the UI and trying to create an Aux image we are getting the following error:
JDK version is 11.0.15.1+2-LTS-10 JAVA_HOME = /usr/java/jdk-11.0.15.1 CLASSPATH = /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/lib/weblogic-deploy-core.jar:/u01/Middleware/wlserver/server/lib/weblogic.jar JAVA_PROPERTIES = -Djava.util.logging.config.class=oracle.weblogic.deploy.logging.WLSDeployLoggingConfig -Dpython.cachedir.skip=true -Dpython.console= /usr/java/jdk-11.0.15.1/bin/java -cp /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/lib/weblogic-deploy-core.jar:/u01/Middleware/wlserver/server/lib/weblogic.jar -Djava.util.logging.config.class=oracle.weblogic.deploy.logging.WLSDeployLoggingConfig -Dpython.cachedir.skip=true -Dpython.console= -Dpython.path=/u01/Middleware/wlserver/common/wlst/modules/jython-modules.jar/Lib org.python.util.jython /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/lib/python/prepare_model.py -oracle_home /u01/Middleware -model_file /home/opc/edea-oke-prod-models/model.yaml -output_dir /home/opc/prepareModelPrNofw -target wko
<Jun 28, 2022 3:40:33 PM> <> <> <The prepareModel program will write its log to /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/logs/prepareModel.log>
<Jun 28, 2022 3:40:35 PM> <The WebLogic Deploy Tooling prepareModel version is 2.1.0:.a158563:Feb 18, 2022 19:09 UTC>
<Jun 28, 2022 3:40:35 PM> <> <No patches home at inventory/patches>
<Jun 28, 2022 3:40:35 PM> <Using configuration file /opt/WebLogic Kubernetes Toolkit UI/tools/wdt-config/targets/wko/target.json>
<Jun 28, 2022 3:40:35 PM> <> <No patches home at /u01/Middleware/inventory/patches>
<Jun 28, 2022 3:40:35 PM> <Using configuration file /opt/WebLogic Kubernetes Toolkit UI/tools/wdt-config/targets/wko/target.json>
<Jun 28, 2022 3:40:36 PM> <Using configuration file /opt/WebLogic Kubernetes Toolkit UI/tools/wdt-config/targets/wko/target.json>
<Jun 28, 2022 3:40:36 PM> <_apply_filter> <Applying filter with ID "wko_filter">
<Jun 28, 2022 3:40:36 PM> <_apply_filter_and_inject_variable>
<Jun 28, 2022 3:40:36 PM> <Using configuration file /opt/WebLogic Kubernetes Toolkit UI/tools/wdt-config/targets/wko/target.json>
<Jun 28, 2022 3:40:36 PM> <Will inject variable replacement strings into model using keyword directions found in file /opt/WebLogic Kubernetes Toolkit UI/tools/wdt-config/targets/wko/target.json>
<Jun 28, 2022 3:40:37 PM>
<Jun 28, 2022 3:40:37 PM> <Using configuration file /opt/WebLogic Kubernetes Toolkit UI/tools/wdt-config/targets/wko/target.json>
<Jun 28, 2022 3:40:37 PM> <_apply_filter> <Applying filter with ID "wko_filter">
<Jun 28, 2022 3:40:38 PM> <_substitute> <Could not resolve secret token "weblogic-credentials:username". Known secrets include: . Secrets can only be resolved in Kubernetes environment. The secret tokens name and keys must have a matching Kubernetes secret name and literals in the same namespace. For WebLogic Kubernetes Operator deployment, you must specify the secret name in "domain.spec.configuration.secrets">
<Jun 28, 2022 3:40:38 PM> <_substitute> <Could not resolve secret token "weblogic-credentials:password". Known secrets include: . Secrets can only be resolved in Kubernetes environment. The secret tokens name and keys must have a matching Kubernetes secret name and literals in the same namespace. For WebLogic Kubernetes Operator deployment, you must specify the secret name in "domain.spec.configuration.secrets">
<Jun 28, 2022 3:40:38 PM> <Using configuration file /opt/WebLogic Kubernetes Toolkit UI/tools/wdt-config/targets/wko/target.json>
<Jun 28, 2022 3:40:38 PM> <_apply_filter> <Applying filter with ID "wko_filter">
<Jun 28, 2022 3:40:38 PM> <_create_file> <Creating target configuration file /home/opc/prepareModelPrNofw/wko-domain.yaml>
Issue Log for prepareModel version 2.1.0 running WebLogic version 14.1.1.0.0 offline mode:
Total: WARNING : 0 SEVERE : 0
prepareModel.sh completed successfully (exit code = 0) /opt/WebLogic Kubernetes Toolkit UI/tools/weblogic-deploy/bin/prepareModel.sh exited with exit code 0 [INFO ] WebLogic Image Tool version 1.11.0 [INFO ] Image Tool build ID: c85d3beb-322f-4a37-8e07-f2b5ea3b1791 [INFO ] Temporary directory used for image build context: /tmp/wlsimgbuilder_temp2770645632245691960 [INFO ] Copying /home/opc/edea-oke-prod-models/model.yaml to build context folder. [INFO ] Copying /home/opc/Downloads/weblogic-deploy.tar.gz to build context folder. [INFO ] Starting build: /usr/bin/podman build --no-cache --force-rm --tag lhr.ocir.io/lrtg4a4iixwu/cfl-prod-reg-aux-image:4 /tmp/wlsimgbuilder_temp2770645632245691960 java.io.IOException: Build command failed with error: Error: kernel does not support overlay fs: unable to create kernel-style whiteout: operation not permitted
/opt/WebLogic Kubernetes Toolkit UI/tools/imagetool/bin/imagetool.sh exited with exit code 0