selinux: provisioning policy customizations

salesgroup commented 4 years ago

We are starting the migration from CoreOS to Fedora CoreOS. No documentation (from Fedora CoreOS) about changing the SSH ports. So we tried this: https://coreos.com/os/docs/latest/customizing-sshd.html#changing-the-sshd-port-1

file: /etc/systemd/system/sshd.socket.d/10-sshd-listen-ports.conf

[Socket] ListenStream= ListenStream=12221

But we got this error: systemd[14685]: sshd.socket: Failed to create listening socket ([::]:12221): Permission denied

Seems related with SElinux.

How can we change the sshd port?

bgilbert commented 4 years ago

CoreOS Container Linux used systemd socket activation for sshd, so the listening port was configured via systemd. Fedora CoreOS runs sshd in the traditional way, as a service that's started automatically, so you'd configure the port in /etc/ssh/sshd_config.

This FCC will do half of the work:

variant: fcos
version: 1.0.0
storage:
  files:
    - path: /etc/ssh/sshd_config
      append:
        - inline: |
            Port 2222

But in addition, sshd_config has this comment:

# If you want to change the port on a SELinux system, you have to tell
# SELinux about this change.
# semanage port -a -t ssh_port_t -p tcp #PORTNUMBER

And we don't ship semanage.

salesgroup commented 4 years ago

I had already tried this but it gave the same type of error.

I think that i've to configure SELinux to give access to sshd to this port. How can i do that?

jlebon commented 4 years ago

I think today this requires modifying the binary policy, which is going to be an issue. I'm not sure if there's a way to only change the runtime value so we can play the same "modify on boot" trick. We can look into that, but we should really get together with the SELinux folks to get this fixed properly.

salesgroup commented 4 years ago

Since "semanage" is not available, how can we do it after server boot?

dustymabe commented 4 years ago

@salesgroup I don't think we have a good answer for you right now. We'd have to figure out how to do what semanage does without running semanage. We have some gaps here on how to manipulate selinux policy in a way that makes more sense for Fedora CoreOS. Ideally any changes to selinux policy would be able to be configured via plain text files that are laid down using Ignition.

Possible workarounds (all are bad):

You might be able to bindmount things in and run semanage from a container
You may choose to just package layer the policycoreutils-python-utils rpm (though see #400)
A really hacky answer may be to run socat or something to redirect traffic from external port 12221 traffic to port 22 locally and block port external 22 using a firewall config

None of those are great answers. We have a similar problem with selinux booleans. As a group we need to talk to the SELinux team and find a path forward for the "lay down config via plain text files" approach.

dustymabe commented 4 years ago

@salesgroup do any of those proposed workaround unblock you?

icedream commented 4 years ago

I have also stumbled across this issue since I need to move the SSH server away from port 22 - it was already in use for Git SSH on a legacy CoreOS installation.

I decided to follow the workaround of running semanage from a container. For this I wrote a script that is installed into CoreOS during Ignition phase and is then run by systemd right before the SSH server starts up via a service dropin. I also explicitly disabled the systemd socket activation to let sshd directly listen on the port.

I have uploaded the actual script, systemd dropin and resulting ignition config parts to this gist.

bgilbert commented 4 years ago

@icedream Thanks for the example! Fedora CoreOS, unlike Container Linux, doesn't use socket activation for sshd, so enabling sshd.service / masking sshd.socket shouldn't be necessary.

remoe commented 3 years ago

For completeness the hacky way (idea from: https://github.com/coreos/fedora-coreos-tracker/issues/396#issuecomment-592028415 ) using proxying 2222 to 22:

    - name: ssh-proxy.service
      enabled: true
      contents: |
        [Unit]
        Description=SSH Proxy
        Wants=network-online.target
        After=network-online.target

        [Install]
        WantedBy=multi-user.target

        [Service]
        Type=notify
        KillMode=mixed
        TimeoutStartSec=0
        Restart=always
        RestartSec=5s
        ExecStart=/usr/bin/socat TCP4-LISTEN:2222,reuseaddr,fork TCP4:127.0.0.1:22

coreos / fedora-coreos-tracker

selinux: provisioning policy customizations #396