search

coreos / fedora-coreos-tracker

Issue tracker for Fedora CoreOS
https://fedoraproject.org/coreos/
262 stars 59 forks source link

Permission denied, when trying to use insmod from privileged container #1018

Open dkhachyan opened 2 years ago

dkhachyan commented 2 years ago

Describe the bug Permission denied, when trying to use insmod from privileged container

Reproduction steps Steps to reproduce the behavior:

  1. Podman run < any privileged container>
  2. Download https://pkg.linbit.com//downloads/drbd/9/drbd-9.1.4.tar.gz?_ga=2.254512143.664777936.1636545525-1528359000.1636545525
  3. Build drbd kernel module
  4. Insmod dbrd kernel module

Expected behavior Kernel module successfully inserted

Actual behavior insmod drbd.ko insmod: ERROR: could not insert module drbd.ko: Permission denied

System details

Additional information Kernel Module injection worked in Fedora Core 33, but we have this issue after upgrade to Fedora Core 34

Build log:

Need a git checkout to regenerate drbd/.drbd_git_revision
make[1]: Entering directory '/tmp/pkg/drbd-9.1.4/drbd'

    Calling toplevel makefile of kernel source tree, which I believe is in
    KDIR=/opt/src/5.13.12-200.fc34.x86_64

make -C /opt/src/5.13.12-200.fc34.x86_64   M=/tmp/pkg/drbd-9.1.4/drbd  modules
  COMPAT  __vmalloc_has_2_params
  COMPAT  alloc_workqueue_takes_fmt
  COMPAT  before_4_13_kernel_read
  COMPAT  blkdev_issue_zeroout_discard
  COMPAT  can_include_vermagic_h
  COMPAT  genl_policy_in_ops
  COMPAT  have_BIO_MAX_VECS
  COMPAT  have_CRYPTO_TFM_NEED_KEY
  COMPAT  have_SHASH_DESC_ON_STACK
  COMPAT  have_WB_congested_enum
  COMPAT  have_allow_kernel_signal
  COMPAT  have_bdi_cap_stable_writes
  COMPAT  have_bdi_congested_fn
  COMPAT  have_bio_bi_bdev
  COMPAT  have_bio_bi_error
  COMPAT  have_bio_bi_opf
  COMPAT  have_bio_bi_status
  COMPAT  have_bio_clone_fast
  COMPAT  have_bio_op_shift
  COMPAT  have_bio_set_dev
  COMPAT  have_bio_set_op_attrs
  COMPAT  have_bio_start_io_acct
  COMPAT  have_bioset_init
  COMPAT  have_bioset_need_bvecs
  COMPAT  have_blk_alloc_queue_rh
  COMPAT  have_blk_check_plugged
  COMPAT  have_blk_qc_t_make_request
  COMPAT  have_blk_queue_flag_set
  COMPAT  have_blk_queue_make_request
  COMPAT  have_blk_queue_merge_bvec
  COMPAT  have_blk_queue_plugged
  COMPAT  have_blk_queue_split_bio
  COMPAT  have_blk_queue_split_q_bio
  COMPAT  have_blk_queue_split_q_bio_bioset
  COMPAT  have_blk_queue_update_readahead
  COMPAT  have_blk_queue_write_cache
  COMPAT  have_d_inode
  COMPAT  have_fallthrough
  COMPAT  have_generic_start_io_acct_q_rw_sect_part
  COMPAT  have_generic_start_io_acct_rw_sect_part
  COMPAT  have_genl_family_parallel_ops
  COMPAT  have_hd_struct
  COMPAT  have_ib_cq_init_attr
  COMPAT  have_ib_get_dma_mr
  COMPAT  have_idr_is_empty
  COMPAT  have_inode_lock
  COMPAT  have_ktime_to_timespec64
  COMPAT  have_kvfree
  COMPAT  have_max_send_recv_sge
  COMPAT  have_nla_nest_start_noflag
  COMPAT  have_nla_parse_deprecated
  COMPAT  have_nla_put_64bit
  COMPAT  have_nla_strscpy
  COMPAT  have_part_stat_h
  COMPAT  have_part_stat_read_accum
  COMPAT  have_pointer_backing_dev_info
  COMPAT  have_proc_create_single
  COMPAT  have_queue_flag_stable_writes
  COMPAT  have_rb_declare_callbacks_max
  COMPAT  have_refcount_inc
  COMPAT  have_req_flush
  COMPAT  have_req_hardbarrier
  COMPAT  have_req_noidle
  COMPAT  have_req_nounmap
  COMPAT  have_req_op_write
  COMPAT  have_req_op_write_same
  COMPAT  have_req_op_write_zeroes
  COMPAT  have_req_prio
  COMPAT  have_req_write
  COMPAT  have_req_write_same
  COMPAT  have_revalidate_disk_size
  COMPAT  have_sched_set_fifo
  COMPAT  have_security_netlink_recv
  COMPAT  have_sendpage_ok
  COMPAT  have_set_capacity_and_notify
  COMPAT  have_shash_desc_zero
  COMPAT  have_simple_positive
  COMPAT  have_sock_set_keepalive
  COMPAT  have_struct_bvec_iter
  COMPAT  have_struct_kernel_param_ops
  COMPAT  have_struct_size
  COMPAT  have_submit_bio
  COMPAT  have_submit_bio_noacct
  COMPAT  have_tcp_sock_set_cork
  COMPAT  have_tcp_sock_set_nodelay
  COMPAT  have_tcp_sock_set_quickack
  COMPAT  have_time64_to_tm
  COMPAT  have_timer_setup
  COMPAT  have_void_make_request
  COMPAT  ib_alloc_pd_has_2_params
  COMPAT  ib_device_has_ops
  COMPAT  ib_post_send_const_params
  COMPAT  ib_query_device_has_3_params
  COMPAT  need_make_request_recursion
  COMPAT  part_stat_read_takes_block_device
  COMPAT  queue_limits_has_discard_zeroes_data
  COMPAT  rdma_create_id_has_net_ns
  COMPAT  sock_create_kern_has_five_parameters
  COMPAT  sock_ops_returns_addr_len
  UPD     /tmp/pkg/drbd-9.1.4/drbd/compat.5.13.12-200.fc34.x86_64.h
  UPD     /tmp/pkg/drbd-9.1.4/drbd/compat.h
  GENPATCHNAMES   5.13.12-200.fc34.x86_64
  SPATCH   777658e4b6258dc8b6250e42870ccfaf  5.13.12-200.fc34.x86_64
  PATCH
patching file drbd_receiver.c
patching file drbd_nl.c
patching file drbd_main.c
patching file drbd_debugfs.c
patching file drbd-headers/linux/genl_magic_func.h
Hunk #2 succeeded at 312 (offset -20 lines).
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_dax_pmem.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_debugfs.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_bitmap.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_proc.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_sender.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_receiver.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_req.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_actlog.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/lru_cache.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_main.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_strings.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_nl.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_interval.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_state.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd-kernel-compat/drbd_wrappers.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_nla.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_transport.o
  GEN     /tmp/pkg/drbd-9.1.4/drbd/drbd_buildtag.c 
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_buildtag.o
  LD [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd.o
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_transport_tcp.o
  MODPOST /tmp/pkg/drbd-9.1.4/drbd/Module.symvers
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd.mod.o
  LD [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd.ko
  BTF [M] /tmp/pkg/drbd-9.1.4/drbd/drbd.ko
Skipping BTF generation for /tmp/pkg/drbd-9.1.4/drbd/drbd.ko due to unavailability of vmlinux
  CC [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_transport_tcp.mod.o
  LD [M]  /tmp/pkg/drbd-9.1.4/drbd/drbd_transport_tcp.ko
  BTF [M] /tmp/pkg/drbd-9.1.4/drbd/drbd_transport_tcp.ko
Skipping BTF generation for /tmp/pkg/drbd-9.1.4/drbd/drbd_transport_tcp.ko due to unavailability of vmlinux
mv .drbd_kernelrelease.new .drbd_kernelrelease
Memorizing module configuration ... done.
make[1]: Leaving directory '/tmp/pkg/drbd-9.1.4/drbd'

    Module build was successful.
=======================================================================
  With DRBD module version 8.4.5, we split out the management tools
  into their own repository at https://github.com/LINBIT/drbd-utils
  (tarball at http://links.linbit.com/drbd-download)

  That started out as "drbd-utils version 8.9.0",
  has a different release cycle,
  and provides compatible drbdadm, drbdsetup and drbdmeta tools
  for DRBD module versions 8.3, 8.4 and 9.

  Again: to manage DRBD 9 kernel modules and above,
  you want drbd-utils >= 9.3 from above url.
=======================================================================
insmod: ERROR: could not insert module ./drbd.ko: Permission denied
insmod: ERROR: could not insert module ./drbd_transport_tcp.ko: Permission denied

Could not load DRBD kernel modules

Systemd log: 

worker-1 audit[109560]: AVC avc:  denied  { module_load } for  pid=109560 comm="insmod" path="/tmp/drbd.ko" dev="tmpfs" ino=6248 scontext=system_u:system_r:spc_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=system permissive=0
Jun 23 15:01:11 worker-1 kernel: audit: type=1400 audit(1624460471.557:725): avc:  denied  { module_load } for  pid=109560 comm="insmod" path="/tmp/drbd.ko" dev="tmpfs" ino=6248 scontext=system_u:system_r:spc_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=system permissive=0
Jun 23 15:01:11 worker-1 audit[109560]: SYSCALL arch=c000003e syscall=313 success=no exit=-13 a0=3 a1=5652517dca2a a2=0 a3=3 items=0 ppid=108768 pid=109560 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=4294967295 comm="insmod" exe="/usr/bin/kmod" subj=system_u:system_r:spc_t:s0 key=(null)
Jun 23 15:01:11 worker-1 kernel: audit: type=1300 audit(1624460471.557:725): arch=c000003e syscall=313 success=no exit=-13 a0=3 a1=5652517dca2a a2=0 a3=3 items=0 ppid=108768 pid=109560 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=4294967295 comm="insmod" exe="/usr/bin/kmod" subj=system_u:system_r:spc_t:s0 key=(null)
Jun 23 15:01:11 worker-1 kernel: audit: type=1327 audit(1624460471.557:725): proctitle=696E736D6F6400647262642E6B6F
Jun 23 15:01:11 worker-1 audit: PROCTITLE proctitle=696E736D6F6400647262642E6B6F
lucab commented 2 years ago

Thanks for the report. It looks like SELinux is blocking this, I'm not sure if it is on purpose or just a slip in the policy. Which FCOS version is this? The version string in your report is not really a valid version, so I can't go back and trace the actual policy package version.

Also, we are in process of rebasing to F35, with our testing and next streams already switched. Can you please check whether the F35 policy also block this? If so, we'll have to forward this report to the policy maintainer and, if they agree, get the fix in F35.

Overall, I'm unsure about this "insmod from container tmpfs" flow versus just placing the build artifacts somewhere under /usr/local/lib/modules and the let the usual module loading machinery take care of it.

cgwalters commented 2 years ago

It may also work to chcon --reference=/usr/lib/modules drbd.ko

dkhachyan commented 2 years ago

We did additional research: 34.20210808.3.0 - has problems with SELinux 34.20210919.3.0 - module injection works fine

Unfortunately we could not check F35 because it seems DRBD module does not support linux kernel 5.14.14

Copying or changing SELinux has no effect

dustymabe commented 2 years ago

Here's the list of everything that changed from 34.20210808.3.0 to 34.20210919.3.0:

$ rpm-ostree --repo=./ db diff 6b7e18634c32d15efd59b18c0fdc5c26d2e00dbd3afc16965509ba6f1c42c274  6334e11901fba7403600522286d817b520eeb15b66b341b392e0c08ed2576b74
ostree diff commit from: 6b7e18634c32d15efd59b18c0fdc5c26d2e00dbd3afc16965509ba6f1c42c274
ostree diff commit to:   6334e11901fba7403600522286d817b520eeb15b66b341b392e0c08ed2576b74
Upgraded:
  audit-libs 3.0.3-1.fc34 -> 3.0.5-1.fc34
  bind-libs 32:9.16.19-1.fc34 -> 32:9.16.20-3.fc34
  bind-license 32:9.16.19-1.fc34 -> 32:9.16.20-3.fc34
  bind-utils 32:9.16.19-1.fc34 -> 32:9.16.20-3.fc34
  bsdtar 3.5.1-2.fc34 -> 3.5.2-2.fc34
  btrfs-progs 5.13.1-1.fc34 -> 5.14-2.fc34
  c-ares 1.17.1-2.fc34 -> 1.17.2-1.fc34
  container-selinux 2:2.164.1-1.git563ba3f.fc34 -> 2:2.167.0-1.fc34
  containerd 1.5.3-1.fc34 -> 1.5.5-1.fc34
  containernetworking-plugins 1.0.0-0.2.rc1.fc34 -> 1.0.0-1.fc34
  coreos-installer 0.10.0-1.fc34 -> 0.10.0-2.fc34
  coreos-installer-bootinfra 0.10.0-1.fc34 -> 0.10.0-2.fc34
  cracklib 2.9.6-25.fc34 -> 2.9.6-27.fc34
  crun 0.20.1-1.fc34 -> 1.0-1.fc34
  dnsmasq 2.85-3.fc34 -> 2.86-1.fc34
  efi-filesystem 5-2.fc34 -> 5-4.fc34
  ethtool 2:5.13-1.fc34 -> 2:5.14-1.fc34
  fedora-release-common 34-1 -> 34-37
  fedora-release-coreos 34-1 -> 34-37
  fedora-release-identity-coreos 34-1 -> 34-37
  fuse-overlayfs 1.7.0-1.fc34 -> 1.7.1-2.fc34
  glib2 2.68.3-1.fc34 -> 2.68.4-1.fc34
  json-glib 1.6.2-1.fc34 -> 1.6.6-1.fc34
  kernel 5.13.7-200.fc34 -> 5.13.16-200.fc34
  kernel-core 5.13.7-200.fc34 -> 5.13.16-200.fc34
  kernel-modules 5.13.7-200.fc34 -> 5.13.16-200.fc34
  krb5-libs 1.19.1-14.fc34 -> 1.19.2-2.fc34
  libarchive 3.5.1-2.fc34 -> 3.5.2-2.fc34
  libipa_hbac 2.5.2-1.fc34 -> 2.5.2-2.fc34
  libmodulemd 2.13.0-1.fc34 -> 2.13.0-2.fc34
  libpwquality 1.4.4-2.fc34 -> 1.4.4-6.fc34
  libsmbclient 2:4.14.6-0.fc34 -> 2:4.14.7-0.fc34
  libsss_certmap 2.5.2-1.fc34 -> 2.5.2-2.fc34
  libsss_idmap 2.5.2-1.fc34 -> 2.5.2-2.fc34
  libsss_nss_idmap 2.5.2-1.fc34 -> 2.5.2-2.fc34
  libsss_sudo 2.5.2-1.fc34 -> 2.5.2-2.fc34
  libuv 1:1.41.0-1.fc34 -> 1:1.42.0-2.fc34
  libwbclient 2:4.14.6-0.fc34 -> 2:4.14.7-0.fc34
  libxcrypt 4.4.24-1.fc34 -> 4.4.25-1.fc34
  linux-firmware 20210716-121.fc34 -> 20210818-122.fc34
  linux-firmware-whence 20210716-121.fc34 -> 20210818-122.fc34
  moby-engine 20.10.7-1.fc34 -> 20.10.8-1.fc34
  mozjs78 78.12.0-1.fc34 -> 78.13.0-1.fc34
  nftables 1:0.9.8-2.fc34 -> 1:0.9.8-3.fc34
  openssl 1:1.1.1k-1.fc34 -> 1:1.1.1l-1.fc34
  openssl-libs 1:1.1.1k-1.fc34 -> 1:1.1.1l-1.fc34
  ostree 2021.3-1.fc34 -> 2021.4-2.fc34
  ostree-libs 2021.3-1.fc34 -> 2021.4-2.fc34
  podman 3:3.2.3-2.fc34 -> 3:3.3.1-1.fc34
  podman-plugins 3:3.2.3-2.fc34 -> 3:3.3.1-1.fc34
  rpm-ostree 2021.7-1.fc34 -> 2021.10-2.fc34
  rpm-ostree-libs 2021.7-1.fc34 -> 2021.10-2.fc34
  samba-client-libs 2:4.14.6-0.fc34 -> 2:4.14.7-0.fc34
  samba-common 2:4.14.6-0.fc34 -> 2:4.14.7-0.fc34
  samba-common-libs 2:4.14.6-0.fc34 -> 2:4.14.7-0.fc34
  selinux-policy 34.14-1.fc34 -> 34.19-1.fc34
  selinux-policy-targeted 34.14-1.fc34 -> 34.19-1.fc34
  shadow-utils 2:4.8.1-8.fc34 -> 2:4.8.1-9.fc34
  skopeo 1:1.3.1-1.fc34 -> 1:1.4.1-1.fc34
  slirp4netns 1.1.9-1.fc34 -> 1.1.12-2.fc34
  sssd-ad 2.5.2-1.fc34 -> 2.5.2-2.fc34
  sssd-client 2.5.2-1.fc34 -> 2.5.2-2.fc34
  sssd-common 2.5.2-1.fc34 -> 2.5.2-2.fc34
  sssd-common-pac 2.5.2-1.fc34 -> 2.5.2-2.fc34
  sssd-ipa 2.5.2-1.fc34 -> 2.5.2-2.fc34
  sssd-krb5 2.5.2-1.fc34 -> 2.5.2-2.fc34
  sssd-krb5-common 2.5.2-1.fc34 -> 2.5.2-2.fc34
  sssd-ldap 2.5.2-1.fc34 -> 2.5.2-2.fc34
  toolbox 0.0.99.2-1.fc34 -> 0.0.99.2-7.fc34
  vim-minimal 2:8.2.3290-1.fc34 -> 2:8.2.3404-1.fc34
Removed:
  firewalld-filesystem-0.9.4-1.fc34.noarch
Added:
  cracklib-dicts-2.9.6-27.fc34.x86_64

If you want to narrow it down further you can test with some of the builds in between 34.20210808.3.0 and 34.20210919.3.0.

ghost commented 2 years ago

I've got same problem.

CoreOS 34.20210919.3.0, kernel 5.13.16-200.fc34.x86_64 - drbd module loaded successfully CoreOS 35.20211029.3.0, kernel 5.14.14-300.fc35.x86_64 - drdb module injection error. (selinux related)

travier commented 2 years ago

@servsav Have you tried the suggestion from https://github.com/coreos/fedora-coreos-tracker/issues/1018#issuecomment-965665248?