Closed ganto closed 8 months ago
I can't even start the daemon... When I do try, this is what I get in the system logs:
Dec 28 21:15:16 mini.server.lan setroubleshoot[77166]: SELinux is preventing systemd from create access on the sock_file unix.socket. For complete SELinux messages run: sealert -l e1d45937-7747-4bba-b67b-a3f0fde5dcb4
Dec 28 21:15:16 mini.server.lan setroubleshoot[77166]: SELinux is preventing systemd from create access on the sock_file unix.socket.
***** Plugin catchall_labels (83.8 confidence) suggests *******************
If you want to allow systemd to have create access on the unix.socket sock_file
Then you need to change the label on unix.socket
Do
# semanage fcontext -a -t FILE_TYPE 'unix.socket'
where FILE_TYPE is one of the following: NetworkManager_dispatcher_console_var_run_t, NetworkManager_var_run_t, abrt_retrace_spool_t, abrt_var_run_t, aiccu_var_run_t, ajaxterm_var_run_t, alsa_var_run_t, antivirus_var_run_t, apcupsd_var_run_t, apmd_var_run_t, arpwatch_var_run_t, asterisk_spool_t, asterisk_var_run_t, audisp_var_run_t, audit_spool_t, auditd_var_run_t, automount_var_run_t, avahi_var_run_t, bacula_var_run_t, bcfg2_var_run_t, bitlbee_var_run_t, blkmapd_var_run_t, blktap_var_run_t, blueman_var_run_t, bluetooth_var_run_t, boltd_var_lib_t, boltd_var_run_t, boothd_var_run_t, bootloader_var_run_t, bootupd_var_run_t, brltty_var_run_t, bumblebee_var_run_t, cache_home_t, cachefilesd_var_run_t, callweaver_var_run_t, canna_var_run_t, cardmgr_var_run_t, ccs_var_run_t, certmaster_var_run_t, certmonger_var_run_t, cgred_var_run_t, chronyd_var_run_t, cinder_var_run_t, clogd_var_run_t, cluster_var_run_t, clvmd_var_run_t, cmirrord_var_run_t, collectd_var_run_t, comsat_var_run_t, condor_var_run_t, config_home_t, conman_var_run_t, conntrackd_var_run_t, consolekit_var_run_t, container_kvm_var_run_t, container_plugin_var_run_t, container_var_run_t, couchdb_var_run_t, courier_spool_t, courier_var_run_t, cpuplug_var_run_t, cpuspeed_var_run_t, cron_spool_t, cron_var_run_t, crond_var_run_t, ctdbd_var_run_t, cupsd_config_var_run_t, cupsd_lpd_var_run_t, cupsd_var_run_t, cvs_var_run_t, cyphesis_var_run_t, cyrus_var_run_t, data_home_t, dbskkd_var_run_t, dbus_home_t, dcc_var_run_t, dccd_var_run_t, dccifd_var_run_t, dccm_var_run_t, dcerpcd_var_run_t, ddclient_var_run_t, deltacloudd_var_run_t, devicekit_var_run_t, devlog_t, dhcpc_var_run_t, dhcpd_var_run_t, dictd_var_run_t, dirsrv_snmp_var_run_t, dirsrv_var_run_t, dkim_milter_data_t, dlm_controld_var_run_t, dnsmasq_var_run_t, dnssec_trigger_var_run_t, dovecot_spool_t, dovecot_var_run_t, drbd_var_run_t, dspam_var_run_t, entropyd_var_run_t, eventlogd_var_run_t, evtchnd_var_run_t, exim_spool_t, exim_var_run_t, fail2ban_var_run_t, fcoemon_var_run_t, fenced_var_run_t, fetchmail_var_run_t, fingerd_var_run_t, firewalld_var_run_t, foghorn_var_run_t, freeipmi_bmc_watchdog_var_run_t, freeipmi_ipmidetectd_var_run_t, freeipmi_ipmiseld_var_run_t, fsadm_var_run_t, fsdaemon_var_run_t, ftpd_var_run_t, games_srv_var_run_t, gconf_home_t, gdomap_var_run_t, getty_var_run_t, gfs_controld_var_run_t, gkeyringd_gnome_home_t, glance_var_run_t, glusterd_var_run_t, gnome_home_t, gnome_initial_setup_var_run_t, gpm_var_run_t, gpsd_var_run_t, greylist_milter_data_t, groupd_var_run_t, gssproxy_var_run_t, gstreamer_home_t, haproxy_var_run_t, hostapd_var_run_t, httpd_var_run_t, hwloc_var_run_t, ibacm_var_run_t, icc_data_home_t, icecast_var_run_t, ifconfig_var_run_t, inetd_child_var_run_t, inetd_var_run_t, init_tmp_t, init_var_lib_t, init_var_run_t, initrc_var_run_t, innd_var_run_t, insights_client_var_run_t, install_var_run_t, ipmievd_var_run_t, ipsec_mgmt_var_run_t, ipsec_var_run_t, iptables_var_lib_t, iptables_var_run_t, irqbalance_var_run_t, iscsi_var_run_t, isnsd_var_run_t, iwhd_var_run_t, jetty_var_run_t, kadmind_var_run_t, keepalived_var_run_t, keystone_var_run_t, kismet_var_run_t, klogd_var_run_t, kmod_var_run_t, krb5kdc_var_run_t, ksmtuned_var_run_t, l2tpd_var_run_t, lircd_var_run_t, lldpad_var_run_t, locate_var_run_t, logwatch_var_run_t, lpd_var_run_t, lsassd_var_run_t, lsmd_var_run_t, lttng_sessiond_var_run_t, lvm_var_run_t, lwiod_var_run_t, lwregd_var_run_t, lwsmd_var_run_t, mail_spool_t, mailman_var_run_t, mcelog_var_run_t, mdadm_var_run_t, memcached_var_run_t, minidlna_var_run_t, minissdpd_var_run_t, mirrormanager_var_run_t, mock_var_run_t, mon_statd_var_run_t, mongod_var_run_t, motion_var_run_t, mount_var_run_t, mpd_var_run_t, mqueue_spool_t, mrtg_var_run_t, mscan_var_run_t, munin_var_run_t, mysqld_var_run_t, mysqlmanagerd_var_run_t, naemon_var_run_t, nagios_spool_t, nagios_var_run_t, named_var_run_t, netlogond_var_run_t, neutron_var_run_t, news_spool_t, ninfod_run_t, nmbd_var_run_t, nova_var_run_t, nrpe_var_run_t, nscd_var_run_t, nsd_var_run_t, nslcd_var_run_t, ntop_var_run_t, ntpd_var_run_t, numad_var_run_t, nut_var_run_t, nx_server_var_run_t, oddjob_var_run_t, opafm_var_run_t, openct_var_run_t, opendnssec_var_run_t, openhpid_var_run_t, openshift_var_run_t, openvpn_var_run_t, openvswitch_var_run_t, openwsman_run_t, osad_var_run_t, pads_var_run_t, pam_var_console_t, pam_var_run_t, passenger_var_run_t, pasta_pid_t, pcp_var_run_t, pcscd_var_run_t, pdns_var_run_t, pegasus_openlmi_storage_var_run_t, pegasus_var_run_t, pesign_var_run_t, piranha_fos_var_run_t, piranha_lvs_var_run_t, piranha_pulse_var_run_t, piranha_web_var_run_t, pkcs11proxyd_var_run_t, pkcs_slotd_var_run_t, pki_ra_var_run_t, pki_tomcat_var_run_t, pki_tps_var_run_t, plymouthd_spool_t, plymouthd_var_run_t, policykit_var_run_t, polipo_pid_t, portmap_var_run_t, portreserve_var_run_t, postfix_spool_bounce_t, postfix_spool_t, postfix_var_run_t, postgresql_var_run_t, postgrey_spool_t, postgrey_var_run_t, pppd_var_run_t, pptp_var_run_t, prelude_audisp_var_run_t, prelude_lml_var_run_t, prelude_spool_t, prelude_var_run_t, print_spool_t, privoxy_var_run_t, prosody_var_run_t, psad_var_run_t, ptal_var_run_t, pulseaudio_var_run_t, puppet_var_run_t, pwauth_var_run_t, pyicqt_var_run_t, pyicqt_var_spool_t, qatlib_var_run_t, qdiskd_var_run_t, qemu_var_run_t, qmail_spool_t, qpidd_var_run_t, quota_nld_var_run_t, rabbitmq_var_run_t, radiusd_var_run_t, radvd_var_run_t, readahead_var_run_t, redis_var_run_t, regex_milter_data_t, restorecond_var_run_t, rhcd_var_run_t, rhev_agentd_var_run_t, rhnsd_var_run_t, rhsmcertd_var_run_t, ricci_modcluster_var_run_t, ricci_var_run_t, rlogind_var_run_t, rngd_var_run_t, roundup_var_run_t, rpcbind_var_run_t, rpcd_var_run_t, rpm_var_run_t, rrdcached_var_run_t, rsync_var_run_t, rtas_errd_var_run_t, rwho_spool_t, sanlock_var_run_t, saslauthd_var_run_t, sbd_var_run_t, sblim_var_run_t, screen_var_run_t, sendmail_var_run_t, sensord_var_run_t, setrans_var_run_t, setroubleshoot_var_run_t, slapd_var_run_t, slpd_var_run_t, smbd_var_run_t, smokeping_var_run_t, smsd_var_run_t, snmpd_var_run_t, snort_var_run_t, sosreport_var_run_t, soundd_var_run_t, spamass_milter_data_t, spamd_spool_t, spamd_var_run_t, spc_var_run_t, squid_var_run_t, squirrelmail_spool_t, srvsvcd_var_run_t, sshd_var_run_t, sslh_var_run_t, sssd_public_t, sssd_var_lib_t, sssd_var_run_t, stalld_var_run_t, stapserver_var_run_t, stratisd_var_run_t, stunnel_var_run_t, svnserve_var_run_t, swat_var_run_t, swift_var_run_t, syslogd_var_run_t, system_cron_spool_t, system_cronjob_var_run_t, system_dbusd_var_run_t, systemd_bootchart_var_run_t, systemd_importd_var_run_t, systemd_logind_inhibit_var_run_t, systemd_logind_sessions_t, systemd_logind_var_run_t, systemd_machined_var_run_t, systemd_networkd_var_run_t, systemd_passwd_var_run_t, systemd_resolved_var_run_t, systemd_timedated_var_run_t, systemd_userdbd_runtime_t, tangd_cache_t, targetclid_var_run_t, telnetd_var_run_t, tftpd_var_run_t, tgtd_var_run_t, thin_aeolus_configserver_var_run_t, thin_var_run_t, timemaster_var_run_t, tlp_var_run_t, tmpfs_t, tomcat_var_run_t, tor_var_run_t, tuned_var_run_t, udev_var_run_t, uml_switch_var_run_t, usbmuxd_var_run_t, user_cron_spool_t, user_tmp_t, useradd_var_run_t, uucpd_spool_t, uucpd_var_run_t, uuidd_var_run_t, var_run_t, var_spool_t, varnishd_var_run_t, varnishlog_var_run_t, vdagent_var_run_t, vhostmd_var_run_t, virt_common_var_run_t, virt_lxc_var_run_t, virt_qemu_ga_var_run_t, virt_var_run_t, virtlogd_var_run_t, vmware_host_pid_t, vmware_pid_t, vnstatd_var_run_t, vpnc_var_run_t, watchdog_var_run_t, wdmd_var_run_t, winbind_rpcd_var_run_t, winbind_var_run_t, xdm_spool_t, xdm_var_run_t, xenconsoled_var_run_t, xend_var_run_t, xenstored_var_run_t, xserver_var_run_t, ypbind_var_run_t, yppasswdd_var_run_t, ypserv_var_run_t, ypxfr_var_run_t, zabbix_var_run_t, zarafa_deliver_var_run_t, zarafa_gateway_var_run_t, zarafa_ical_var_run_t, zarafa_indexer_var_run_t, zarafa_monitor_var_run_t, zarafa_server_var_run_t, zarafa_spooler_var_run_t, zebra_var_run_t, zoneminder_var_run_t.
Then execute:
restorecon -v 'unix.socket'
***** Plugin catchall (17.1 confidence) suggests **************************
If you believe that systemd should be allowed create access on the unix.socket sock_file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'systemd' --raw | audit2allow -M my-systemd
# semodule -X 300 -i my-systemd.pp
Yes, I ran my analysis with SELinux in "permissive" mode. Your log is the more verbose version of the first alert that I posted for the startup.
As a starting point. We can relocate the socket to /run/ and set the INCUS_SOCKET environment variable to it. That will get the socket started and seems to be the workaround sorted out for lxd a few years ago
https://github.com/containers/container-selinux
Seems to be where lxd has the file contexts defined. Lxc has file contexts in their repo. How does lxd get properly labeled in the build process?
I seem to have incus working with SELinux right now.
I had to use the same workarounds for LXD socket.
Using the following file_contexts:
/usr/s?bin/incus -- gen_context(system_u:object_r:container_runtime_exec_t:s0)
/usr/s?bin/incus-.* -- gen_context(system_u:object_r:container_runtime_exec_t:s0)
/usr/lib/incus/.* -- gen_context(system_u:object_r:container_runtime_exec_t:s0)
/usr/lib/systemd/system/incus.* -- gen_context(system_u:object_r:container_unit_file_t:s0)
/var/lib/incus(/.*)? gen_context(system_u:object_r:container_var_lib_t:s0)
/var/log/incus(/.*)? gen_context(system_u:object_r:container_log_t:s0)
I have a working alpine container created. It has networking once modifying firewalld.
VM's also appear to work fine.
First I tried to create a SELinux policy with the file context provided by @m2Giles then I tried to summarize them as:
/usr/s?bin/incus(.*)? gen_context(system_u:object_r:container_runtime_exec_t:s0)
/usr/lib/systemd/system/incus.* gen_context(system_u:object_r:container_unit_file_t:s0)
/var/lib/incus(/.*)? gen_context(system_u:object_r:container_var_lib_t:s0)
/var/log/incus(/.*)? gen_context(system_u:object_r:container_log_t:s0)
Unfortunately in both cases I still get an error during the %selinux_modules_install
of an incus-selinux
package:
[...]
Running scriptlet: container-selinux-2:2.226.0-1.fc39.noarch 6/27
Installing : container-selinux-2:2.226.0-1.fc39.noarch 6/27
Running scriptlet: container-selinux-2:2.226.0-1.fc39.noarch 6/27
Running scriptlet: incus-selinux-0.4-0.4.fc39.noarch 7/27
Installing : incus-selinux-0.4-0.4.fc39.noarch 7/27
Running scriptlet: incus-selinux-0.4-0.4.fc39.noarch 7/27
Failed to resolve filecon statement at /var/lib/selinux/targeted/tmp/modules/200/incus/cil:2
Failed to resolve AST
/usr/sbin/semodule: Failed!
[...]
container-selinux
is installed as dependency. Will continue investigating next time.
Source code used for testing can be found in the incus-selinux
branch.
For your summarized one of them is missing.
/usr/lib/incus/(.*)
Unsure why it's failing. I was able to use semanage fcontext to set each of these.
This is what I have set via semanage context:
$ sudo semanage fcontext -l | grep "incus"
/usr/bin/incus all files system_u:object_r:container_runtime_exec_t:s0
/usr/lib/incus/.* all files system_u:object_r:container_runtime_exec_t:s0
/usr/lib/systemd/system/incus.* all files system_u:object_r:container_unit_file_t:s0
/usr/s?bin/incus all files system_u:object_r:container_runtime_exec_t:s0
/usr/s?bin/incus-.* all files system_u:object_r:container_runtime_exec_t:s0
/var/lib/incus/(/.*)? all files system_u:object_r:container_var_lib_t:s0
/var/log/incus/(/.*)? all files system_u:object_r:container_log_t:s0
So far I've yet to hit an SELinux denials.
The SELinux should now be fixed. Still have to update the documentation and fix the Rawhide builds. But feedback is welcome.
Thank you for getting this working. Besides a small workaround needed on my silverblue derived system (issue with how ostree manages file contexts), it works! I'm especially grateful you got the socket working in the correct location as well.
Any chance that you would share this workaround? I think there are also other people using Silverblue (e.g. #37) that might profit from it.
The issue is that /usr stuff won't have the right file contexts following install due to a bug with how ostree manages file contexts. They will exist in SELinux's compiled list so a restorecon
will fix the problem but /usr is read-only. I work around this using a systemd service that will fix it on each boot.
# /etc/systemd/system/incus-workaround.service
[Unit]
Description=Workaround SELinux issues with Incus...
ConditionPathExists=/usr/bin/incus
ConditionPathExists=/usr/bin/incus-agent
ConditionPathExists=/usr/lib/incus
After=local-fs.target
[Service]
Type=oneshot
ExecStart=/usr/bin/mount --bind -o rw /usr/bin/incus /usr/bin/incus
ExecStart=/usr/bin/mount --bind -o rw /usr/bin/incus-agent /usr/bin/incus-agent
ExecStart=/usr/bin/mount --bind -o rw /usr/lib/incus /usr/lib/incus
ExecStart=/usr/sbin/restorecon -R -v /usr/bin/incus
ExecStart=/usr/sbin/restorecon -R -v /usr/bin/incus-agent
ExecStart=/usr/sbin/restorecon -R -v /usr/lib/incus
ExecStart=/usr/bin/umount /usr/bin/incus
ExecStart=/usr/bin/umount /usr/bin/incus-agent
ExecStart=/usr/bin/umount /usr/lib/incus
RemainAfterExit=true
[Install]
WantedBy=multi-user.target
Something similar is necessary for swtpm
due to the same issue with ostree if you are using virtual machines thaopent need the tpm device.
Additionally, silverblue won't relabel things by default, if a user already was using incus, they will need to relabel /var/{lib,log}/incus
. This would only be necessary if they already existed.
# restorecon -R -v /var/lib/incus
# restorecon -R -v /var/log/incus
The socket works with no workarounds needed which is awesome.
shell» incus admin init --minimal
Error: Failed to connect to local daemon: Get "http://unix.socket/1.0": dial unix /var/lib/incus/unix.socket: connect: permission denied
I can see that the service is running:
shell» systemctl status incus
● incus.service - Incus - Daemon
Loaded: loaded (/usr/lib/systemd/system/incus.service; indirect; preset: disabled)
Drop-In: /usr/lib/systemd/system/service.d
└─10-timeout-abort.conf
Active: active (running) since Wed 2024-01-17 14:11:45 GMT; 5min ago
TriggeredBy: ● incus.socket
Docs: man:incusd(1)
Process: 18382 ExecStartPost=/usr/lib/incus/incusd waitready --timeout=600 (code=exited, status=0/SUCCESS)
Main PID: 18381 (incusd)
Tasks: 28
Memory: 109.0M
CPU: 335ms
CGroup: /system.slice/incus.service
└─18381 /usr/lib/incus/incusd --group incus-admin
Jan 17 14:11:44 myserver.lan systemd[1]: Starting incus.service - Incus - Daemon...
Jan 17 14:11:44 myserver.lan incusd[18381]: time="2024-01-17T14:11:44Z" level=warning msg="AppArmor support has been disabled because of lack of kernel suppor>
Jan 17 14:11:44 myserver.lan incusd[18381]: time="2024-01-17T14:11:44Z" level=warning msg=" - AppArmor support has been disabled, Disabled because of lack of >
Jan 17 14:11:44 myserver.lan incusd[18381]: time="2024-01-17T14:11:44Z" level=warning msg=" - Couldn't find the CGroup hugetlb controller, hugepage limits wil>
Jan 17 14:11:44 myserver.lan incusd[18381]: time="2024-01-17T14:11:44Z" level=warning msg=" - Couldn't find the CGroup network priority controller, per-instan>
Jan 17 14:11:45 myserver.lan systemd[1]: Started incus.service - Incus - Daemon.
Version:
shell» incus --version
0.4
Could it be that I just need to wait for the new package? :)
Are you a member of the incus-admin group?
Ah, right... Tried with superuser:
» sudo incus admin init --minimal
[sudo] password for bmll:
Error: Failed to create local member network "incusbr0" in project "default": Failed generating auto config: Failed to automatically find an unused IPv4 subnet, manual configuration required
I will try to do the manual configuration, as suggested. Thanks @m2Giles !
OK. I found what caused the above (in case someone else is having similar problem). - Incus, just like lxd/lxc, tries to setup a 10.x.x.x network by default, when none is available, like it is in my case (10.0.0.0/8 via 172.2.2.1 dev tun0
.. long story why it is so), it fails. So, I created manually a "smaller" 172.16.xx.xx network (192.168.16.x would also work for an example), and it all worked, well kind of.
So, something like the following helped: incus network create incusbr0 --type bridge ipv4.address=192.168.10.1/24
However incus containers can't get IPv4 addresses. At least I could create containers... :)
It turned out that firewalld was enabled and it caused all this...
Various operations of Incus violate the default SELinux policies present in Fedora.
systemctl start incus
(unconfigured):incus admin init --minimal
:incus launch images:ubuntu/22.04 ubuntu-container
:Measured on a Fedora 39 host with
selinux-policy-39.3-1.fc39
andincus-0.4
.