Support OpenVPN3 on Fedora Silverblue

[p5]

As discussed in #171, we will move Fedora Silverblue (and similar ostree-based OSes) to a separate issue as there are some additional things we need to work around.

Right now using the development build provided by dsommers, we are receiving a fatal error from DBus after trying to establish a connection to the server.

Summary of current issues:

1. Since we are installing OpenVPN3 inside the OCI build, /var is not writable and therefore /var/lib/openvpn3/configs is not created, causing an error. A workaround is to manually create this directory in a booted system.
2. DBus fatal error when starting VPN session (see below).

Error:

``` Jan 20 13:09:27 fedora openvpn3-service-logger[7196]: {tag:13500574870050280773} Configuration override 'log-level' set to '6' Jan 20 13:09:28 fedora openvpn3-service-logger[7196]: {tag:3261999402174143353} Cleaning up resources for PID 22996. Jan 20 13:09:28 fedora openvpn3-service-logger[7196]: {tag:13500574870050280773} Starting connection Jan 20 13:09:28 fedora openvpn3-service-logger[7196]: {tag:13500574870050280773} Waiting for server response Jan 20 13:09:28 fedora openvpn3-service-backendstart-dev[22996]: ** ERROR ** Failed closing down D-Bus connection: [Proxy::Client('net.openvpn.v3.log', '/net/openvpn/v3/log', 'net.openvpn.v3.log')] DBus::Connection is not valid** ERROR ** An unrecoveralbe D-Bus error occured Jan 20 13:09:28 fedora openvpn3-service-backendstart-dev[22996]: openvpn3-service-client lost the 'net.openvpn.v3.backends.be22996' registration on the D-Bus Jan 20 13:09:28 fedora openvpn3-service-backendstart-dev[22996]: [DBus::Connection] Connection flush failed: The connection is closed Jan 20 13:09:33 fedora openvpn3-service-logger[7196]: Detached: {tag:16607725973821941176} [:1.216/net.openvpn.v3.backends], pid 22989 Jan 20 13:09:33 fedora systemd[1]: dbus-:1.3-net.openvpn.v3.backends@2.service: Deactivated successfully. ```

I will try and provide some additional information if I can find it out.

[p5]

I believe the crux of your issue is related to these two log lines:

Jan 20 13:09:28 fedora openvpn3-service-backendstart-dev[22996]: ** ERROR **  Failed closing down D-Bus connection: [Proxy::Client('net.openvpn.v3.log', '/net/openvpn/v3/log', 'net.openvpn.v3.log')] DBus::Connection is not valid** ERROR **  An unrecoveralbe D-Bus error occured
Jan 20 13:09:28 fedora openvpn3-service-backendstart-dev[22996]:              openvpn3-service-client lost the 'net.openvpn.v3.backends.be22996' registration on the D-Bus

The busname the openvpn3-service-client uses to register on the D-Bus (net.openvpn.v3.backends.be22996) is supposed to be unique (and it carries the process PID in the bus name). That the D-Bus broker kicks it off the bus instantly is unexpected. It could be related to the D-Bus policies not being in the right location for Silverblue (in particular /usr/share/dbus-1/system.d/net.openvpn.v3.client.conf). It could also be related to some challenges related to dynamic UID/GID values (I believe Silverblue uses that a lot more than ordinary Fedora distros). Silverblue uses quite different ways to build, install and update a distribution, which openvpn3-linux might not yet have accounted for.

Source: https://github.com/OpenVPN/openvpn3-linux/issues/171#issuecomment-1902747674

[dsommers]

Thank you for creating this ticket. I'll pay attention to this and will help out as needed. I can't promise we will manage any needed fixes for the next v22_dev release, but this will get priority after that release. Any help here in the meantime will surely be appreciated.

Some debug tricks for this issue. Since it seems to be related to openvpn3-service-client stopping way too early, there are few ways to trace this down.

First, enable the openvpn user account so you can run processes as this user easily from a terminal.

a) Inspecting D-Bus communication

Become the openvpn user and start the openvpn3-service-backendstart manually:

  openvpn@host:~ $ /usr/libexec/openvpn3-linux/openvpn3-service-backendstart --log-level 6 --idle-exit 0 --client-log-level 6 --client-log-file stdout: --client-setenv G_DBUS_DEBUG=all

b) Step-by-step start of a VPN session, to attach gdb in the early phase

I don't know how familiar you are with such low-level debugging, so I'll add more details just to be sure. First install the needed packages:

 user@host:~ $  sudo dnf install gdb dnf-utils
 [...]
 user@host:~ $  sudo debuginfo-install openvpn3 openvpn3-client gdbuspp glib2

Start the VPN session manually. You need the D-Bus object path of the configuration profile to do this. Use openvpn3 configs-list -v to see the D-Bus paths for profiles already imported with openvpn3 config-import. Use a VPN configuration profile not requiring any username/password/2fa authentication; that will complicate these steps (I won't go through that here now)

Initiate the VPN session:

  user@host:~ $ gdbus call --system --dest net.openvpn.v3.sessions --object-path /net/openvpn/v3/sessions --method net.openvpn.v3.sessions.NewTunnel '/net/openvpn/v3/configuration/f5645ac5x6138x4718xad98x7fd836f86493'

The /net/openvpn/v3/configuration/f5645ac5x6138x4718xad98x7fd836f86493 string is the D-Bus object path to the configuration profile. This must be changed to your own path.

Use openvpn3 sessions-list to see the PID value of the openvpn3-service-client process. If this failed (no PID value available), then it really stopped too early to easily attach debugger. If you have a PID value:

 user@host:~ $ sudo gdb --pid 11332
 [....]
 (gdb)  catch throw
 Catchpoint 1 (throw)
 (gdb)  continue
 Continuing.

Now the debugger is attached, and the next steps can be attempted. In another terminal run this command:

 $ gdbus call --system --dest net.openvpn.v3.sessions --object-path /net/openvpn/v3/sessions/82695e2as6ddes4f20sae38s878395de0af8 --method net.openvpn.v3.sessions.Ready 
 ()
 $ gdbus call --system --dest net.openvpn.v3.sessions --object-path /net/openvpn/v3/sessions/82695e2as6ddes4f20sae38s878395de0af8 --method net.openvpn.v3.sessions.Connect 
 ()

If the client process at some point crashed or threw an exception, you can use the thread apply all bt command in gdb to get a backtrace of what the application did before stopping. That output is the details needed to further debug where it failed.

This should hopefully get you started retrieving more details on where things halt on Silverblue.

[dsommers]

I also forgot, you can start all the background services manually in a similar way to openvpn3-service-backendstart. The -backendstart is responsible for kicking off the openvpn3-service-client processes, so that is the only one you can't start manually.

All these processes with the exception of the openvpn3-service-netcfg process is expected to be started as the openvpn user. The -netcfg service must start as root, as it need to preserve a few privileges (like CAP_NET_ADMIN) to be able to perform its tasks. Once all privileges are set up, it will drop the rest and switch to the openvpn user.

Another thing to test is also to run the system with SELinux in permissive mode. Also inspect the /var/log/audit/audit.log for denied log lines mentioning openvpn3* processes.

[p5]

This might be far simpler than expected. I completely forgot about SELinux, so tried setting it to permissive and retrying the connection, and it worked :tada: Was able to telnet an internal resource from the Silverblue host through a container (because Fedora doesn't ship with telnet).

OpenVPN3 with OpenVPN Cloud + SSO + split tunneling worked! This was both with and without sudo privileges.

So I guess we may just need to work out the correct SELinux settings? (I say this knowing nothing about SELinux, so happy to be told it's more work)

Here are some logs I think may be relevant:

❯ sudo ausearch -m AVC,USER_AVC,SELINUX_ERR,USER_SELINUX_ERR -ts recent
----
time->Mon Jan 22 12:33:10 2024
type=AVC msg=audit(1705926790.102:578): avc:  denied  { read write } for  pid=923 comm="dbus-broker" path="/dev/net/tun" dev="devtmpfs" ino=403 scontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:tun_tap_device_t:s0 tclass=chr_file permissive=1
----
time->Mon Jan 22 12:35:22 2024
type=AVC msg=audit(1705926922.947:635): avc:  denied  { read write } for  pid=923 comm="dbus-broker" path="/dev/net/tun" dev="devtmpfs" ino=403 scontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:tun_tap_device_t:s0 tclass=chr_file permissive=1

(Note: This is still using the development build)

[dsommers]

Wow! That's awesome!

That denial looks like something which there should already be an SELinux policy for.

Can you check that the openvpn3-selinux package is installed? And then check if the dbus_access_tuntap_device boolean setting is enabled?

 # getsebool dbus_access_tuntap_device
 dbus_access_tuntap_device --> on

If not, then you need to run this command:

 # setsebool -P dbus_access_tuntap_device=1

[p5]

The openvpn3-selinux package is installed

❯ rpm -qa | grep openvpn
openvpn-2.6.8-1.fc39.x86_64
NetworkManager-openvpn-1.10.2-3.fc39.x86_64
NetworkManager-openvpn-gnome-1.10.2-3.fc39.x86_64
openvpn3-selinux-21-4.gdbusdev1.fc39.noarch
openvpn3-21-4.gdbusdev1.fc39.x86_64
openvpn3-client-21-4.gdbusdev1.fc39.x86_64

dbus_access_tuntap_device is not enabled on my machine.

❯ getsebool dbus_access_tuntap_device
dbus_access_tuntap_device --> off

After running setsebool -P dbus_access_tuntap_device=1, re-enforcing SELinux and rebooting, the connection is still unsuccessful.

Note: Now that I've enabled dbus_access_tuntap_device, I am not receiving any SELinux errors. Maybe there is more going on than I initially thought... Will continue troubleshooting

Error logs:

``` an 22 14:39:24 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dbus-:1.3-net.openvpn.v3.backends@3 comm="systemd" exe="/u> Jan 22 14:39:24 fedora openvpn3-service-backendstart-dev[13786]: openvpn3-linux dev_gdbuspp-migration:a9befb5d8d1eaa03 (openvpn3-service-backendstart-dev) Jan 22 14:39:24 fedora openvpn3-service-backendstart-dev[13786]: OpenVPN core 3.git:HEAD:ea747cba linux x86_64 64-bit Jan 22 14:39:24 fedora openvpn3-service-backendstart-dev[13786]: Copyright (C) 2012-2022 OpenVPN Inc. All rights reserved. Jan 22 14:39:24 fedora openvpn3-service-logger[6064]: Detached: {tag:4425683552010626967} [:1.202/net.openvpn.v3.backends] Jan 22 14:39:24 fedora openvpn3-service-logger[6064]: Detached: {tag:17581907637226695787} [:1.202/net.openvpn.v3.sessions] Jan 22 14:39:24 fedora openvpn3-service-logger[6064]: Attached: {tag:12829758549375769082} [:1.205/net.openvpn.v3.backends], pid 13786 Jan 22 14:39:24 fedora openvpn3-service-backendstart-dev[13792]: Re-initiated process from pid 13792 to backend process pid 13793 Jan 22 14:39:24 fedora openvpn3-service-backendstart-dev[13793]: openvpn3-linux dev_gdbuspp-migration:a9befb5d8d1eaa03 (openvpn3-service-client-dev) Jan 22 14:39:24 fedora openvpn3-service-backendstart-dev[13793]: OpenVPN core 3.git:HEAD:ea747cba linux x86_64 64-bit Jan 22 14:39:24 fedora openvpn3-service-backendstart-dev[13793]: Copyright (C) 2012-2022 OpenVPN Inc. All rights reserved. Jan 22 14:39:24 fedora openvpn3-service-logger[6064]: Attached: {tag:5764075039462540765} [:1.206/net.openvpn.v3.backends], pid 13793 Jan 22 14:39:24 fedora openvpn3-service-logger[6064]: Attached: {tag:8682382985116571369} [:1.206/net.openvpn.v3.sessions], pid 13793 Jan 22 14:39:24 fedora openvpn3-service-logger[6064]: Assigned session /net/openvpn/v3/sessions/043b2de8seda9s4367sbe12s7e98a17ce1c4 to {tag:5764075039462540765} Jan 22 14:39:24 fedora openvpn3-service-logger[6064]: {tag:5764075039462540765} Configuration override 'log-level' set to '6' Jan 22 14:39:25 fedora openvpn3-service-logger[6064]: {tag:11371020398077427138} Cleaning up resources for PID 13793. Jan 22 14:39:25 fedora openvpn3-service-logger[6064]: {tag:5764075039462540765} Starting connection Jan 22 14:39:25 fedora openvpn3-service-logger[6064]: {tag:5764075039462540765} Waiting for server response Jan 22 14:39:25 fedora openvpn3-service-backendstart-dev[13793]: ** ERROR ** Failed closing down D-Bus connection: [Proxy::Client('net.openvpn.v3.log', '/net/openvpn/v3/log', 'net.openvpn.v3> Jan 22 14:39:25 fedora openvpn3-service-backendstart-dev[13793]: openvpn3-service-client lost the 'net.openvpn.v3.backends.be13793' registration on the D-Bus Jan 22 14:39:25 fedora openvpn3-service-backendstart-dev[13793]: [DBus::Connection] Connection flush failed: The connection is closed ```

[dsommers]

dbus_access_tuntap_device is not enabled on my machine.

That's odd, as that should have happened during the openvpn3-selinux package install. So this might be related to Silverblue restrictions. I'm using several RPM packaging macros (%selinux_set_booleans), so need to inspect what they do.

[dsommers]

Might also be worth checking the output of semanage module -l | grep openvpn3 ... Two policies should be listed, openvpn3 and openvpn3_service.

[p5]

Might also be worth checking the output of semanage module -l | grep openvpn3 ... Two policies should be listed, openvpn3 and openvpn3_service.

Correct, two policies are listed

❯ sudo semanage module -l | grep openvpn3
openvpn3                  200       pp    
openvpn3_service          200       pp 

I'm going back to your original comment with the dbus debugging steps

[dsommers]

Just a quick thought appeared as I looked at the openvpn3.spec file.

Can you, as root, run openvpn3-admin init-config ... what does that report?

[p5]

Just a quick thought appeared as I looked at the openvpn3.spec file.

Can you, as root, run openvpn3-admin init-config ... what does that report?

~ 
❯ sudo su
[sudo] password for fedora: 

root in /var/home/fedora 
❯ openvpn3-admin init-config
- Detected settings will be saved to disk? No

* Checking for OpenVPN user and group accounts
    Found:  openvpn user - uid 982
    Found:  openvpn group - gid 978

* Checking OpenVPN 3 Linux state/configuration directory
    Using directory: /var/lib/openvpn3
    Directory found

* Logger Configuration
    Configuration file: /var/lib/openvpn3/log-service.json
    systemd-journald active state: active
    :: Result ::  Will use systemd journald for logging
    !! Configuration UNCHANGED

* Network Configuration
    Configuration file: /var/lib/openvpn3/netcfg.json
    Found systemd-resolved
    Found accessible /etc/resolv.conf
    Parsing /etc/resolv.conf ... Done
    Found systemd-resolved configured (127.0.0.53) in /etc/resolv.conf 
    :: Result :: Will use systemd-resolved
    !! Configuration UNCHANGED

* Ensuring SELinux file labels are correct
    - SELinux status: Enforcing mode

[dsommers]

That generally looks good. If those .json files are owned by and writeable by the openvpn user and group in addition to the /var/lib/openvpn3 directory, then the service configuration aspects should be covered.

[p5]

That generally looks good. If those .json files are owned by and writeable by the openvpn user and group in addition to the /var/lib/openvpn3 directory, then the service configuration aspects should be covered.

Checking this, and they don't actually exist. /var is not writable during installation (technically it is writable, but not persisted to the running system), so I had to run mkdir /var/lib/openvpn3 after booting up the system to get around one error that prevented the VPN start (SELinux permissive).

I guess I may need to manually create those json files too to get it working with SELinux enabled? And longer term, find where to put these.

root in /var/home/fedora 
❯ ls -la /var/lib/openvpn3/
total 0
drwxr-x---. 1 openvpn openvpn  14 Jan 20 13:06 .
drwxr-xr-x. 1 root    root    832 Jan 22 19:26 ..
drwxr-xr-x. 1 openvpn openvpn   0 Jan 20 13:06 configs

root in /var/home/fedora 
❯ ls -la /var/lib/openvpn3/configs/
total 0
drwxr-xr-x. 1 openvpn openvpn  0 Jan 20 13:06 .
drwxr-x---. 1 openvpn openvpn 14 Jan 20 13:06 ..

[dsommers]

This line looks wrong (from ls -la /var/lib/openvpn3) ....

  drwxr-xr-x. 1 root    root    832 Jan 22 19:26 ..

That need to be owned by openvpn:openvpn with 0600, 0640 or 0660

[p5]

This line looks wrong (from ls -la /var/lib/openvpn3) ....

  drwxr-xr-x. 1 root    root    832 Jan 22 19:26 ..

That need to be owned by openvpn:openvpn with 0600, 0640 or 0660

I may be misunderstanding, but I believe this just means /var/lib is owned by root? If I do ls -la /var/lib, the openvpn3 directory is owned by the openvpn:openvpn (see collapsed terminal output). Currently looking how to find the permission code (0600 etc) of the directory

``` ❯ ls -la /var/lib total 20 drwxr-xr-x. 1 root root 832 Jan 22 19:26 . drwxr-xr-x. 1 root root 228 Jan 22 14:50 .. drwxrwxr-x. 1 root root 20 Dec 2 16:37 AccountsService drwxr-xr-x. 1 root root 24 Jan 22 22:59 alsa lrwxrwxrwx. 1 root root 26 Dec 2 16:37 alternatives -> ../../usr/lib/alternatives drwxr-xr-x. 1 root root 0 Dec 2 16:37 authselect drwx------. 1 root root 8 Dec 2 16:37 bluetooth drwxr-xr-x. 1 root root 0 Dec 2 16:37 brltty drwxr-x---. 1 chrony chrony 10 Jan 22 22:59 chrony drwxr-xr-x. 1 root root 16 Dec 2 16:37 cni drwxr-xr-x. 1 root root 6 Dec 2 16:37 color drwxr-xr-x. 1 colord colord 58 Dec 2 16:37 colord drwxr-xr-x. 1 root root 30 Dec 2 16:37 containers drwxr-xr-x. 1 root root 20 Dec 2 16:37 dbus drwxr-xr-x. 1 root root 0 Dec 2 16:37 dhclient drwxr-xr-x. 1 root dnsmasq 0 Dec 2 16:37 dnsmasq drwxr-xr-x. 1 root root 10 Jan 20 01:38 fedora-third-party drwxr-xr-x. 1 root root 136 Jan 24 12:20 flatpak drwx------. 1 root root 0 Dec 2 16:37 fprint drwxr-xr-x. 1 root root 86 Jan 20 01:38 fwupd drwxr-xr-x. 1 root root 0 Dec 2 16:37 games drwxrwx--T. 1 gdm gdm 66 Jan 22 00:51 gdm drwxr-xr-x. 1 geoclue geoclue 0 Dec 2 16:37 geoclue drwxr-xr-x. 1 root root 50 Dec 2 16:37 gssproxy drwxr-xr-x. 1 root root 0 Dec 2 16:37 hp drwx------. 1 apache apache 0 Dec 2 16:37 httpd drwxr-xr-x. 1 root root 0 Dec 2 16:37 hyperv drwxr-xr-x. 1 root root 0 Dec 2 16:37 initramfs drwxr-xr-x. 1 root root 8 Jan 20 01:38 libvirt drwxr-xr-x. 1 root root 32 Jan 24 12:20 logrotate drwx------. 1 root root 0 Jan 20 01:38 machines drwxr-xr-x. 1 root root 0 Dec 2 16:37 misc drwxr-xr-x. 1 root root 46 Dec 2 16:37 net-snmp drwx------. 1 root root 250 Jan 24 12:50 NetworkManager drwxr-xr-x. 1 root root 68 Dec 2 16:47 nfs drwxrwx---. 1 openvpn openvpn 0 Dec 2 16:37 openvpn drwxr-x---. 1 openvpn openvpn 14 Jan 20 13:06 openvpn3 drwxr-xr-x. 1 root root 0 Jan 22 14:50 os-prober drwxr-xr-x. 1 root root 20 Jan 24 12:53 plocate drwxr-xr-x. 1 root root 26 Dec 2 16:37 plymouth drwxr-x---. 1 root polkitd 28 Dec 2 16:37 polkit-1 drwx------. 1 root root 0 Dec 2 16:37 portables drwxr-xr-x. 1 root root 0 Dec 2 16:37 power-profiles-daemon drwx------. 1 root root 36 Jan 22 19:26 private lrwxrwxrwx. 1 root root 19 Dec 2 16:37 rpm -> ../../usr/share/rpm drwxr-xr-x. 1 root root 14 Dec 2 16:43 rpm-ostree lrwxrwxrwx. 1 root root 26 Jan 22 19:26 rpm-ostree-countme -> private/rpm-ostree-countme drwxr-xr-x. 1 root root 12 Dec 2 16:47 rpm-state drwxr-xr-x. 1 root root 22 Dec 2 16:37 samba drwxr-xr-x. 1 root root 32 Dec 2 16:47 selinux drwxr-xr-x. 1 root root 100 Dec 2 16:37 sss drwxr-xr-x. 1 root root 126 Dec 2 16:37 systemd drwxr-xr-x. 1 root root 12 Dec 2 16:37 tpm2-tss drwx------. 1 root root 0 Dec 2 16:37 udisks2 drwxr-xr-x. 1 root root 0 Dec 2 16:37 upower lrwxrwxrwx. 1 root root 21 Dec 2 16:37 vagrant -> ../../usr/lib/vagrant drwxr-xr-x. 1 root root 0 Dec 2 16:37 xkb ```

Edit: Current directory permissions are 0750. Trying to change to one you mentioned previously

[dsommers]

Yeah, this is correct:

 drwxr-x---. 1 openvpn openvpn  14 Jan 20 13:06 [/var/lib/]openvpn3

The openvpn3-service-logger and openvpn3-service-netcfg will write their configurations to disk via openvpn3-admin {log-service, netcfg-service} --config-{set,unset} commands.

[p5]

After chowning from 0750 to 0600, I was unable to connect to the VPN with SELiunx permissive and received permission denied errors on /var/lib/openvpn3/configs

Jan 24 13:13:56 fedora openvpn3-service-configmgr[109742]: Configuration file setup error: Error parsing --state-dir: Cannot open state-dir directory '/var/lib/openvpn3/configs': Permission d>

[dsommers]

Duh! I meant 0700 on the directory. Sorry!

[dsommers]

The openvpn3-service-* ... processes runs as openvpn:openvpn. The openvpn3-service-configmgr uses /var/lib/openvpn3/configs to store persistent configuration profiles. Which is why it needs access to that directory. The other two processes currently using /var/lib/openvpn3 are openvpn3-service-logger and openvpn3-service-netcfg.

[p5]

Ah, with 0700, we're back to a working permissive system and D-Bus errors start showing up again in enforced mode. Note: There's still no json files in /var/lib/openvpn3 for netcfg or log-service.

Jan 24 13:19:42 fedora openvpn3-service-backendstart-dev[134636]: ** ERROR **  Failed closing down D-Bus connection: [Proxy::Client('net.openvpn.v3.log', '/net/openvpn/v3/log', 'net.openvpn.v3.log')] DBus::Connection is not valid** ERROR **  An unrecoveralbe D-Bus error occured
Jan 24 13:19:42 fedora openvpn3-service-backendstart-dev[134636]:              openvpn3-service-client lost the 'net.openvpn.v3.backends.be134636' registration on the D-Bus

[dsommers]

Okay, so it's now currently experienced as a pure SELinux related issue? It connects and works as expected with SELinux in permissive mode but not in enforced mode; is that about right?

[p5]

Okay, so it's now currently experienced as a pure SELinux related issue? It connects and works as expected with SELinux in permissive mode but not in enforced mode; is that about right?

It all appears functional with SELinux in permissive, although I doubt my use-case hits the full functionality of the app. The main issue is when SELinux is enforced. There are a few workarounds required (like manually creating /var/lib/openvpn3/configs) but I'm not too concerned about that now. I'm sure there's an easy fix for this.

Previously (before the glib development build) connecting even with SELinux permissive failed, so there were definitely improvements made with the new builds.

[dsommers]

This is actually quite good news, I would say. If you have had a chance to test user/password authentication, then the most critical code paths should have been good enough tested. The last auth method is web-based authentication, but those code paths aren't really modified (the openURL() call happens in the openvpn3 command).

Thanks a lot so far for all testing. I'll try to get a Silverblue VM running and will debug the SELinux issues. At this point the dontaudit rules needs to disabled (so we do get the dontaudit errors in the audit.log). But you do get lots of noise and filtering which denial entry is right or wrong can be hard. Unless you really want to dig into these aspects, I can poke at it a bit later.

I believe we can manage to squeeze in a fix for this in the v22_dev release - unless a completely new policy is required.

[dsommers]

Regarding:

There are a few workarounds required (like manually creating /var/lib/openvpn3/configs) but I'm not too concerned about that now. I'm sure there's an easy fix for this.

I think I'll make the openvpn3-admin init-config do that. That is being run as a %post install script in the .spec file. That should fix this at install time. If Silverblue allows that.

[p5]

I think I'll make the openvpn3-admin init-config do that. That is being run as a %post install script in the .spec file. That should fix this at install time. If Silverblue allows that.

I don't believe generating these in the %post will resolve the issue. At the end of the installation (inside the OCI container build), /var or /tmp cannot contain anything else the build fails.

You can see the install steps at the PR below: https://github.com/rsturla/eternal-images/pull/15/files#diff-ce06fe0f10235017c2c9b84a52bffa977801f5d24f934f4f78377f5564b9b90aR74-R82

My guess would be to create these files if not exists the first time openvpn is called (or the first action that reads these files). Alternatively the installers, in this case myself, could write a systemd oneshot (or this could be supplied by this package) that runs openvpn3-admin init-config on boot rather than install.

[dsommers]

Hmmm ... I see. Well, using the appropriate way to have these directories persistent from install and done according to Silverblue standards should be the main goal. I don't want us to hack something which may break in the future.

I'm just wondering how other applications which needs persistent data solves this? Are there some Silverblue packaging/composition guides somewhere which can be studied?

If there are better "persistent state directories" to use than /var/lib/openvpn3 ... we can also look at how to do that.

Is there an appropriate mailing list where such things is being discussed? I'm already on the Fedora devel list.

[p5]

Sorry for not responding for two weeks.

I'm hoping to get a Fedora Atomic maintainer in here to help with some of these discussions. Besides the basics (like /var doesn't persist during build and /usr isn't writable at runtime), I don't really have any idea how things should be done.

I believe if /var/lib/openvpn3 contains configuration specific to the system that will be using it, the files would ideally be created at runtime since the machine which does the installs the package would be an ephemeral GitHub Actions runner or similar.
If it's generic, and not tailored to the system, maybe /etc would be the ideal location, or if there's some special location in /usr which I'm unaware of, maybe there.

===

People seem active and willing to help in their #Fedora Atomic Desktops room in the Fedora Matrix, so maybe a good place to start (sharing the image as I honestly have no clue how OpenVPN and/or Atomic desktops work behind the scenes)

[travier]

General packaging rules:

• Use /usr for all the content of the RPM (binaries, default config files, etc.)
• Use /etc for user/admin configs that override the defaults
• Use /var for runtime content

Only the first two are available during package installation.

[dsommers]

No worries, @p5!

I'm considering alternatives to have a more flexible model in regards to /var/lib/openvpn3 as well. Currently, the only place this path is hard-coded is in the net.openvpn.v3.*.service D-Bus auto-start files. And some "OpenVPN 3 Linux generic settings", like the --state-dir argument could indeed be picked up by a file in /etc. But since these services may also update the "runtime state" and wanting it to be preserved through service restarts, it feels like a bit too-far-stretch to update files in /etc (those files should be more static, potentially managed by configuration management toos like Puppet or Chef).

In regards to dynamic UIDs, that should not be an issue. The user and group names are compile-time set to openvpn and it will always do a UID lookup on-the-fly whenever it drops privileges (and similar operations) to the openvpn user.

[dsommers]

@travier

That sounds reasonable (I'm a long time Fedora package maintainer for the openvpn package and a few others). The only thing I'm wondering about is the /var bullet point. You say "runtime content", is that to be understood as a persistent or ephemeral storage location in rpm-ostree context?

Several of the auto-started D-Bus services will store their runtime modified settings in /var/lib/openvpn3. And when using openvpn3 config-import to import a VPN profile with the --persistent flag set, the profile is stored in /var/lib/openvpn3/configs.

[travier]

/var is persistent like on other systems (i.e. it's not a tmpfs).

[bcdady]

Has there been progress on this possibility of supporting (providing packages) for openvpn3 on Fedora Silverblue? I just loaded up BlueFin DX, and cannot get the OS built-in VPN client to connect, nor have I been able to install the openvpn3 (CLI) client.

I've tried following the openvpn-3-client-for-linux as well as tried the community OpenVPN3 Linux documentation.

[dsommers]

@bcdady Unfortunately, not yet - at least not from my side. It is on my todo-list, but unfortunately it's not surfaced as a high-priority issue currently. That said, I'm interested to get this working!

If anyone has some time available to dig into this further, I would appreciate all the help we can get. That's the quickest way to resolve this sooner than waiting on me getting to this.

The challenge is to get something working with the "first runtime setup", which is adding a challenge since /var is not accessible at package install time. The current approach is to run openvpn3-admin init-config --write-configs at package as a post-install script, which configures /var/lib/openvpn3 and writes some initial configuration files there. This needs to be done more clever for these situations. And this challenge is somewhat related to what NixOS has fought with as well in https://github.com/NixOS/nixpkgs/pull/326623.