containers / buildah

A tool that facilitates building OCI images.
https://buildah.io
Apache License 2.0
7.45k stars 785 forks source link

Graalvm fails to load shared library due to selinux #3425

Closed jskov-jyskebank-dk closed 3 years ago

jskov-jyskebank-dk commented 3 years ago

Description

Graalvm java/keytool fails to run in ubi8.4 base image due to selinux.

Steps to reproduce the issue:

  1. Create this Containerfile:
    
    $ cat > Containerfile <<EOF
    FROM registry.access.redhat.com/ubi8:8.4-206.1626828523

ENV GRAALVM_VERSION='21.1.0' \ GRAALVM_SHA256SUM=39252954d2cb16dbc8ce4269f8b93a326a0efffdce04625615e827fe5b5e4ab7 \ JAVA_HOME=/opt/graalvm

RUN curl -L https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-\${GRAALVM_VERSION}/graalvm-ce-java11-linux-amd64-\${GRAALVM_VERSION}.tar.gz -o /tmp/java.tgz \ && echo "\$GRAALVM_SHA256SUM /tmp/java.tgz" | sha256sum -c - \ && mkdir -p \$JAVA_HOME \ && tar -x --strip-components=1 -f /tmp/java.tgz -C \$JAVA_HOME \ && rm -rf /tmp/java.tgz

ENV PATH="\$PATH:\$JAVA_HOME/bin"

RUN java -version RUN keytool -list EOF


2. Build it: podman build -f Containerfile

**Describe the results you received:**

STEP 1: FROM registry.access.redhat.com/ubi8:8.4-206.1626828523 STEP 2: ENV GRAALVM_VERSION='21.1.0' GRAALVM_SHA256SUM=39252954d2cb16dbc8ce4269f8b93a326a0efffdce04625615e827fe5b5e4ab7 JAVA_HOME=/opt/graalvm --> fafda048fc6 STEP 3: RUN curl -L https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-${GRAALVM_VERSION}/graalvm-ce-java11-linux-amd64-${GRAALVM_VERSION}.tar.gz -o /tmp/java.tgz && echo "$GRAALVM_SHA256SUM /tmp/java.tgz" | sha256sum -c - && mkdir -p $JAVA_HOME && tar -x --strip-components=1 -f /tmp/java.tgz -C $JAVA_HOME && rm -rf /tmp/java.tgz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 651 100 651 0 0 2079 0 --:--:-- --:--:-- --:--:-- 2079 100 388M 100 388M 0 0 16.2M 0 0:00:23 0:00:23 --:--:-- 14.8M /tmp/java.tgz: OK --> ed015cd7020 STEP 4: ENV PATH="$PATH:$JAVA_HOME/bin" --> 0e262db31bc STEP 5: RUN java -version openjdk version "11.0.11" 2021-04-20 OpenJDK Runtime Environment GraalVM CE 21.1.0 (build 11.0.11+8-jvmci-21.1-b05) OpenJDK 64-Bit Server VM GraalVM CE 21.1.0 (build 11.0.11+8-jvmci-21.1-b05, mixed mode, sharing) --> 3f1d844feb9 STEP 6: RUN keytool -list #

A fatal error has been detected by the Java Runtime Environment:

#

Internal Error (jvmci.cpp:82), pid=1, tid=13

fatal error: Unable to load JVMCI shared library from /opt/graalvm/lib/libjvmcicompiler.so: /opt/graalvm/lib/libjvmcicompiler.so: cannot restore segment prot after reloc: Permission denied

#

JRE version: OpenJDK Runtime Environment GraalVM CE 21.1.0 (11.0.11+8) (build 11.0.11+8-jvmci-21.1-b05)

Java VM: OpenJDK 64-Bit Server VM GraalVM CE 21.1.0 (11.0.11+8-jvmci-21.1-b05, mixed mode, sharing, tiered, jvmci, jvmci compiler, compressed oops, g1 gc, linux-amd64)

Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" (or dumping to //core.1)

#

An error report file with more information is saved as:

//hs_err_pid1.log

#

If you would like to submit a bug report, please visit:

https://github.com/oracle/graal/issues

#

[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f255b111e91]

Error: error building at STEP "RUN keytool -list": error while running runtime: exit status 127


I have seen similar output from the `java --version` command.

**Describe the results you expected:**

If I disable selinux I get:

```console
$ podman build --security-opt label=disable -f Containerfile
...
STEP 6: RUN keytool -list
keytool error: java.lang.Exception: Keystore file does not exist: /root/.keystore
Error: error building at STEP "RUN keytool -list": error while running runtime: exit status 1

Using Fedora:34 as base image also seems to work, so I am unsure if this is a problem of the base image or my Fedora 33 development box.

The se log shows:

$ sudo ausearch -m avc -ts recent
...
time->Thu Aug  5 08:15:45 2021
type=AVC msg=audit(1628144145.465:1111): avc:  denied  { execmod } for  pid=23558 comm=4A564D43492D6E617469766520436F path="/opt/graalvm/lib/libjvmcicompiler.so" dev="dm-5" ino=8520308 scontext=system_u:system_r:container_t:s0:c567,c770 tcontext=unconfined_u:object_r:usr_t:s0 tclass=file permissive=0

Output of rpm -q buildah or apt list buildah:

buildah-1.21.4-4.fc33.x86_64

Output of buildah version:

Version:         1.21.4
Go Version:      go1.15.13
Image Spec:      1.0.1-dev
Runtime Spec:    1.0.2-dev
CNI Spec:        0.4.0
libcni Version:  
image Version:   5.12.0
Git Commit:      
Built:           Thu Jan  1 01:00:00 1970
OS/Arch:         linux/amd64

Output of podman version if reporting a podman build issue:

Version:      3.2.3
API Version:  3.2.3
Go Version:   go1.15.13
Built:        Fri Jul 16 22:49:57 2021
OS/Arch:      linux/amd64

*Output of `cat /etc/release`:**

Fedora release 33 (Thirty Three)
NAME=Fedora
VERSION="33 (Workstation Edition)"
ID=fedora
VERSION_ID=33
VERSION_CODENAME=""
PLATFORM_ID="platform:f33"
PRETTY_NAME="Fedora 33 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:33"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=33
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=33
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Workstation Edition"
VARIANT_ID=workstation
Fedora release 33 (Thirty Three)
Fedora release 33 (Thirty Three)

Output of uname -a:

Linux localhost.localdomain 5.12.15-200.fc33.x86_64 #1 SMP Wed Jul 7 19:56:49 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Output of cat /etc/containers/storage.conf:

# This file is is the configuration file for all tools
# that use the containers/storage library.
# See man 5 containers-storage.conf for more information
# The "container storage" table contains all of the server options.
[storage]

# Default Storage Driver, Must be set for proper operation.
driver = "overlay"

# Temporary storage location
runroot = "/run/containers/storage"

# Primary Read/Write location of container storage
graphroot = "/var/lib/containers/storage"

# Storage path for rootless users
#
# rootless_storage_path = "$HOME/.local/share/containers/storage"

[storage.options]
# Storage options to be passed to underlying storage drivers

# AdditionalImageStores is used to pass paths to additional Read/Only image stores
# Must be comma separated list.
additionalimagestores = [
]

# Remap-UIDs/GIDs is the mapping from UIDs/GIDs as they should appear inside of
# a container, to the UIDs/GIDs as they should appear outside of the container,
# and the length of the range of UIDs/GIDs.  Additional mapped sets can be
# listed and will be heeded by libraries, but there are limits to the number of
# mappings which the kernel will allow when you later attempt to run a
# container.
#
# remap-uids = 0:1668442479:65536
# remap-gids = 0:1668442479:65536

# Remap-User/Group is a user name which can be used to look up one or more UID/GID
# ranges in the /etc/subuid or /etc/subgid file.  Mappings are set up starting
# with an in-container ID of 0 and then a host-level ID taken from the lowest
# range that matches the specified name, and using the length of that range.
# Additional ranges are then assigned, using the ranges which specify the
# lowest host-level IDs first, to the lowest not-yet-mapped in-container ID,
# until all of the entries have been used for maps.
#
# remap-user = "containers"
# remap-group = "containers"

# Root-auto-userns-user is a user name which can be used to look up one or more UID/GID
# ranges in the /etc/subuid and /etc/subgid file.  These ranges will be partitioned
# to containers configured to create automatically a user namespace.  Containers
# configured to automatically create a user namespace can still overlap with containers
# having an explicit mapping set.
# This setting is ignored when running as rootless.
# root-auto-userns-user = "storage"
#
# Auto-userns-min-size is the minimum size for a user namespace created automatically.
# auto-userns-min-size=1024
#
# Auto-userns-max-size is the minimum size for a user namespace created automatically.
# auto-userns-max-size=65536

[storage.options.overlay]
# ignore_chown_errors can be set to allow a non privileged user running with
# a single UID within a user namespace to run containers. The user can pull
# and use any image even those with multiple uids.  Note multiple UIDs will be
# squashed down to the default uid in the container.  These images will have no
# separation between the users in the container. Only supported for the overlay
# and vfs drivers.
#ignore_chown_errors = "false"

# Path to an helper program to use for mounting the file system instead of mounting it
# directly.
#mount_program = "/usr/bin/fuse-overlayfs"

# mountopt specifies comma separated list of extra mount options
mountopt = "nodev"

# Set to skip a PRIVATE bind mount on the storage home directory.
# skip_mount_home = "false"

# Size is used to set a maximum size of the container image.
# size = ""

# ForceMask specifies the permissions mask that is used for new files and
# directories.
#
# The values "shared" and "private" are accepted.
# Octal permission masks are also accepted.
#
#  "": No value specified.
#     All files/directories, get set with the permissions identified within the
#     image.
#  "private": it is equivalent to 0700.
#     All files/directories get set with 0700 permissions.  The owner has rwx
#     access to the files. No other users on the system can access the files.
#     This setting could be used with networked based homedirs.
#  "shared": it is equivalent to 0755.
#     The owner has rwx access to the files and everyone else can read, access
#     and execute them. This setting is useful for sharing containers storage
#     with other users.  For instance have a storage owned by root but shared
#     to rootless users as an additional store.
#     NOTE:  All files within the image are made readable and executable by any
#     user on the system. Even /etc/shadow within your image is now readable by
#     any user.
#
#   OCTAL: Users can experiment with other OCTAL Permissions.
#
#  Note: The force_mask Flag is an experimental feature, it could change in the
#  future.  When "force_mask" is set the original permission mask is stored in
#  the "user.containers.override_stat" xattr and the "mount_program" option must
#  be specified. Mount programs like "/usr/bin/fuse-overlayfs" present the
#  extended attribute permissions to processes within containers rather then the
#  "force_mask"  permissions.
#
# force_mask = ""

[storage.options.thinpool]
# Storage Options for thinpool

# autoextend_percent determines the amount by which pool needs to be
# grown. This is specified in terms of % of pool size. So a value of 20 means
# that when threshold is hit, pool will be grown by 20% of existing
# pool size.
# autoextend_percent = "20"

# autoextend_threshold determines the pool extension threshold in terms
# of percentage of pool size. For example, if threshold is 60, that means when
# pool is 60% full, threshold has been hit.
# autoextend_threshold = "80"

# basesize specifies the size to use when creating the base device, which
# limits the size of images and containers.
# basesize = "10G"

# blocksize specifies a custom blocksize to use for the thin pool.
# blocksize="64k"

# directlvm_device specifies a custom block storage device to use for the
# thin pool. Required if you setup devicemapper.
# directlvm_device = ""

# directlvm_device_force wipes device even if device already has a filesystem.
# directlvm_device_force = "True"

# fs specifies the filesystem type to use for the base device.
# fs="xfs"

# log_level sets the log level of devicemapper.
# 0: LogLevelSuppress 0 (Default)
# 2: LogLevelFatal
# 3: LogLevelErr
# 4: LogLevelWarn
# 5: LogLevelNotice
# 6: LogLevelInfo
# 7: LogLevelDebug
# log_level = "7"

# min_free_space specifies the min free space percent in a thin pool require for
# new device creation to succeed. Valid values are from 0% - 99%.
# Value 0% disables
# min_free_space = "10%"

# mkfsarg specifies extra mkfs arguments to be used when creating the base
# device.
# mkfsarg = ""

# metadata_size is used to set the `pvcreate --metadatasize` options when
# creating thin devices. Default is 128k
# metadata_size = ""

# Size is used to set a maximum size of the container image.
# size = ""

# use_deferred_removal marks devicemapper block device for deferred removal.
# If the thinpool is in use when the driver attempts to remove it, the driver
# tells the kernel to remove it as soon as possible. Note this does not free
# up the disk space, use deferred deletion to fully remove the thinpool.
# use_deferred_removal = "True"

# use_deferred_deletion marks thinpool device for deferred deletion.
# If the device is busy when the driver attempts to delete it, the driver
# will attempt to delete device every 30 seconds until successful.
# If the program using the driver exits, the driver will continue attempting
# to cleanup the next time the driver is used. Deferred deletion permanently
# deletes the device and all data stored in device will be lost.
# use_deferred_deletion = "True"

# xfs_nospace_max_retries specifies the maximum number of retries XFS should
# attempt to complete IO when ENOSPC (no space) error is returned by
# underlying storage device.
# xfs_nospace_max_retries = "0"
rhatdan commented 3 years ago

Just as a guess, could you update containers-common and execute restorecon -R -v $HOME/.lib/share/containers

jskov-jyskebank-dk commented 3 years ago

Yes!

Well, restorecon did not help (restorecon -R -v $HOME/.local/share/containers) I also tried with sudo (on the realpath).

But your guess means you suspect bad local data. So I deleted $HOME/.local/share/containers. And that did the trick.

I had feared it was due to some selinux configuration that would eventually filter through to coreos on our OpenShift instances. And that would be pretty bleak.

So thanks a lot!