search

perfsonar / toolkit

perfSONAR Toolkit distribution environment scripts and GUI
Apache License 2.0
30 stars 5 forks source link

firewall not set with fresh toolkit install on Debian10 #458

Closed szymontrocha closed 3 months ago

szymontrocha commented 5 months ago

I noticed that after fresh install of perfsonar-toolkit bundle under Debian 10.13 I don't have any perfsonar setting in firewall which I guess should open/block specific ports related to services:

root@psmall-poz1:~# firewall-cmd --list-all
Error: INVALID_ZONE
root@psmall-poz1:~# firewall-cmd --list-all --zone=public
public
  target: default
  icmp-block-inversion: no
  interfaces:
  sources:
  services:
  ports:
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

root@psmall-poz1:~# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
INPUT_direct  all  --  anywhere             anywhere
INPUT_ZONES_SOURCE  all  --  anywhere             anywhere
INPUT_ZONES  all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
FORWARD_direct  all  --  anywhere             anywhere
FORWARD_IN_ZONES_SOURCE  all  --  anywhere             anywhere
FORWARD_IN_ZONES  all  --  anywhere             anywhere
FORWARD_OUT_ZONES_SOURCE  all  --  anywhere             anywhere
FORWARD_OUT_ZONES  all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
OUTPUT_direct  all  --  anywhere             anywhere

Chain INPUT_direct (1 references)
target     prot opt source               destination

Chain INPUT_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain INPUT_ZONES (1 references)
target     prot opt source               destination
IN_public  all  --  anywhere             anywhere            [goto]

Chain FORWARD_direct (1 references)
target     prot opt source               destination

Chain FORWARD_IN_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain FORWARD_IN_ZONES (1 references)
target     prot opt source               destination
FWDI_public  all  --  anywhere             anywhere            [goto]

Chain FORWARD_OUT_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain FORWARD_OUT_ZONES (1 references)
target     prot opt source               destination
FWDO_public  all  --  anywhere             anywhere            [goto]

Chain OUTPUT_direct (1 references)
target     prot opt source               destination

Chain IN_public (1 references)
target     prot opt source               destination
IN_public_log  all  --  anywhere             anywhere
IN_public_deny  all  --  anywhere             anywhere
IN_public_allow  all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere

Chain IN_public_log (1 references)
target     prot opt source               destination

Chain IN_public_deny (1 references)
target     prot opt source               destination

Chain IN_public_allow (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh ctstate NEW,UNTRACKED

Chain FWDI_public (1 references)
target     prot opt source               destination
FWDI_public_log  all  --  anywhere             anywhere
FWDI_public_deny  all  --  anywhere             anywhere
FWDI_public_allow  all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere

Chain FWDI_public_log (1 references)
target     prot opt source               destination

Chain FWDI_public_deny (1 references)
target     prot opt source               destination

Chain FWDI_public_allow (1 references)
target     prot opt source               destination

Chain FWDO_public (1 references)
target     prot opt source               destination
FWDO_public_log  all  --  anywhere             anywhere
FWDO_public_deny  all  --  anywhere             anywhere
FWDO_public_allow  all  --  anywhere             anywhere

Chain FWDO_public_log (1 references)
target     prot opt source               destination

Chain FWDO_public_deny (1 references)
target     prot opt source               destination

Chain FWDO_public_allow (1 references)
target     prot opt source               destination
root@psmall-poz1:~# pscheduler troubleshoot
Performing basic troubleshooting of psmall-poz1.

psmall-poz1:

  Measuring MTU... 65535 (Local)
  Looking for pScheduler... OK.
  Fetching API level... 5
  Checking clock... OK.
  Exercising API... Archivers... Contexts... Tests... Tools... OK.
  Fetching service status... OK.
  Checking services... Ticker... Scheduler... Runner... Archiver... OK.
  Checking limits... OK.
  Idle test.... 5 seconds... Missed... Failed.

Did not get a result: Resource Not found.

root@psmall-poz1:/var/log#
laeti-tia commented 5 months ago

Might be related to #427

laeti-tia commented 5 months ago

@szymontrocha Can you check if enabling the buster-backports repository and installing iptables from it can have an effect on your issue?

This can be done by adding the following line in the /etc/apt/sources.list:

deb http://deb.debian.org/debian buster-backports main

and then installing the new version with:

apt-get install -t buster-backports iptables

And then rerun the perfSONAR script.

szymontrocha commented 5 months ago

I think this seem to change the situation:

# apt-get install -t buster-backports iptables
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libip4tc2 libip6tc2 libnftnl11 libxtables12 netbase
Recommended packages:
  nftables
The following NEW packages will be installed:
  libip4tc2 libip6tc2
The following packages will be upgraded:
  iptables libnftnl11 libxtables12 netbase
4 upgraded, 2 newly installed, 0 to remove and 83 not upgraded.
Need to get 579 kB of archives.
After this operation, 78.8 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://deb.debian.org/debian buster-backports/main amd64 iptables amd64 1.8.5-3~bpo10+1 [384 kB]
Get:2 http://deb.debian.org/debian buster-backports/main amd64 libxtables12 amd64 1.8.5-3~bpo10+1 [44.5 kB]
Get:3 http://deb.debian.org/debian buster-backports/main amd64 libip4tc2 amd64 1.8.5-3~bpo10+1 [34.6 kB]
Get:4 http://deb.debian.org/debian buster-backports/main amd64 libip6tc2 amd64 1.8.5-3~bpo10+1 [34.8 kB]
Get:5 http://deb.debian.org/debian buster-backports/main amd64 netbase all 6.1~bpo10+1 [19.9 kB]
Get:6 http://deb.debian.org/debian buster-backports/main amd64 libnftnl11 amd64 1.1.7-1~bpo10+1 [61.1 kB]
Fetched 579 kB in 0s (3,630 kB/s)
Reading changelogs... Done
(Reading database ... 89219 files and directories currently installed.)
Preparing to unpack .../0-iptables_1.8.5-3~bpo10+1_amd64.deb ...
Unpacking iptables (1.8.5-3~bpo10+1) over (1.8.2-4) ...
Preparing to unpack .../1-libxtables12_1.8.5-3~bpo10+1_amd64.deb ...
Unpacking libxtables12:amd64 (1.8.5-3~bpo10+1) over (1.8.2-4) ...
Selecting previously unselected package libip4tc2:amd64.
Preparing to unpack .../2-libip4tc2_1.8.5-3~bpo10+1_amd64.deb ...
Unpacking libip4tc2:amd64 (1.8.5-3~bpo10+1) ...
Selecting previously unselected package libip6tc2:amd64.
Preparing to unpack .../3-libip6tc2_1.8.5-3~bpo10+1_amd64.deb ...
Unpacking libip6tc2:amd64 (1.8.5-3~bpo10+1) ...
Preparing to unpack .../4-netbase_6.1~bpo10+1_all.deb ...
Unpacking netbase (6.1~bpo10+1) over (5.6) ...
Preparing to unpack .../5-libnftnl11_1.1.7-1~bpo10+1_amd64.deb ...
Unpacking libnftnl11:amd64 (1.1.7-1~bpo10+1) over (1.1.2-2) ...
Setting up libip4tc2:amd64 (1.8.5-3~bpo10+1) ...
Setting up libip6tc2:amd64 (1.8.5-3~bpo10+1) ...
Setting up libnftnl11:amd64 (1.1.7-1~bpo10+1) ...
Setting up libxtables12:amd64 (1.8.5-3~bpo10+1) ...
Setting up netbase (6.1~bpo10+1) ...
Installing new version of config file /etc/services ...
Setting up iptables (1.8.5-3~bpo10+1) ...
Processing triggers for man-db (2.8.5-2) ...
Processing triggers for libc-bin (2.28-10+deb10u2) ...
root@psmall-poz1:/etc/apt# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
f2b-sshd   tcp  --  anywhere             anywhere             multiport dports ssh
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
INPUT_direct  all  --  anywhere             anywhere
INPUT_ZONES_SOURCE  all  --  anywhere             anywhere
INPUT_ZONES  all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
FORWARD_direct  all  --  anywhere             anywhere
FORWARD_IN_ZONES_SOURCE  all  --  anywhere             anywhere
FORWARD_IN_ZONES  all  --  anywhere             anywhere
FORWARD_OUT_ZONES_SOURCE  all  --  anywhere             anywhere
FORWARD_OUT_ZONES  all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
OUTPUT_direct  all  --  anywhere             anywhere

Chain INPUT_direct (1 references)
target     prot opt source               destination

Chain INPUT_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain INPUT_ZONES (1 references)
target     prot opt source               destination
IN_public  all  --  anywhere             anywhere            [goto]

Chain FORWARD_direct (1 references)
target     prot opt source               destination

Chain FORWARD_IN_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain FORWARD_IN_ZONES (1 references)
target     prot opt source               destination
FWDI_public  all  --  anywhere             anywhere            [goto]

Chain FORWARD_OUT_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain FORWARD_OUT_ZONES (1 references)
target     prot opt source               destination
FWDO_public  all  --  anywhere             anywhere            [goto]

Chain OUTPUT_direct (1 references)
target     prot opt source               destination

Chain IN_public (1 references)
target     prot opt source               destination
IN_public_log  all  --  anywhere             anywhere
IN_public_deny  all  --  anywhere             anywhere
IN_public_allow  all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere

Chain IN_public_log (1 references)
target     prot opt source               destination

Chain IN_public_deny (1 references)
target     prot opt source               destination

Chain IN_public_allow (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh ctstate NEW,UNTRACKED

Chain FWDI_public (1 references)
target     prot opt source               destination
FWDI_public_log  all  --  anywhere             anywhere
FWDI_public_deny  all  --  anywhere             anywhere
FWDI_public_allow  all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere

Chain FWDI_public_log (1 references)
target     prot opt source               destination

Chain FWDI_public_deny (1 references)
target     prot opt source               destination

Chain FWDI_public_allow (1 references)
target     prot opt source               destination

Chain FWDO_public (1 references)
target     prot opt source               destination
FWDO_public_log  all  --  anywhere             anywhere
FWDO_public_deny  all  --  anywhere             anywhere
FWDO_public_allow  all  --  anywhere             anywhere

Chain FWDO_public_log (1 references)
target     prot opt source               destination

Chain FWDO_public_deny (1 references)
target     prot opt source               destination

Chain FWDO_public_allow (1 references)
target     prot opt source               destination

Chain f2b-sshd (1 references)
target     prot opt source               destination
REJECT     all  --  103.91.136.18        anywhere             reject-with icmp-port-unreachable
REJECT     all  --  218.92.0.47          anywhere             reject-with icmp-port-unreachable
REJECT     all  --  83.121.168.184.host.secureserver.net  anywhere             reject-with icmp-port-unreachable
REJECT     all  --  91.149.238.71        anywhere             reject-with icmp-port-unreachable
RETURN     all  --  anywhere             anywhere
root@psmall-poz1:/etc/apt# apt --reinstall install perfsonar-toolkit-security
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 7,388 B of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://downloads.perfsonar.net/debian perfsonar-release/main amd64 perfsonar-toolkit-security all 5.0.7-1 [7,388 B]
Fetched 7,388 B in 0s (15.0 kB/s)
(Reading database ... 89231 files and directories currently installed.)
Preparing to unpack .../perfsonar-toolkit-security_5.0.7-1_all.deb ...
Unpacking perfsonar-toolkit-security (5.0.7-1) over (5.0.7-1) ...
Setting up perfsonar-toolkit-security (5.0.7-1) ...
Adding perfSONAR firewall rules
root@psmall-poz1:/etc/apt# firewall-cmd --list-all
public
  target: default
  icmp-block-inversion: no
  interfaces:
  sources:
  services: bwctl dhcpv6-client http https ntp owamp-control ssh traceroute twamp-control
  ports: 8760-9960/udp 8760-9960/tcp 18760-19960/udp 18760-19960/tcp 5201/tcp 5201/udp 5001/tcp 5001/udp 5000/tcp 5101/tcp 5000/udp 5101/udp 5890-5900/tcp
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

root@psmall-poz1:/etc/apt# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
INPUT_direct  all  --  anywhere             anywhere
INPUT_ZONES_SOURCE  all  --  anywhere             anywhere
INPUT_ZONES  all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
FORWARD_direct  all  --  anywhere             anywhere
FORWARD_IN_ZONES_SOURCE  all  --  anywhere             anywhere
FORWARD_IN_ZONES  all  --  anywhere             anywhere
FORWARD_OUT_ZONES_SOURCE  all  --  anywhere             anywhere
FORWARD_OUT_ZONES  all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
OUTPUT_direct  all  --  anywhere             anywhere

Chain INPUT_direct (1 references)
target     prot opt source               destination

Chain INPUT_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain INPUT_ZONES (1 references)
target     prot opt source               destination
IN_public  all  --  anywhere             anywhere            [goto]

Chain FORWARD_direct (1 references)
target     prot opt source               destination

Chain FORWARD_IN_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain FORWARD_IN_ZONES (1 references)
target     prot opt source               destination
FWDI_public  all  --  anywhere             anywhere            [goto]

Chain FORWARD_OUT_ZONES_SOURCE (1 references)
target     prot opt source               destination

Chain FORWARD_OUT_ZONES (1 references)
target     prot opt source               destination
FWDO_public  all  --  anywhere             anywhere            [goto]

Chain OUTPUT_direct (1 references)
target     prot opt source               destination

Chain IN_public (1 references)
target     prot opt source               destination
IN_public_log  all  --  anywhere             anywhere
IN_public_deny  all  --  anywhere             anywhere
IN_public_allow  all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere

Chain IN_public_log (1 references)
target     prot opt source               destination

Chain IN_public_deny (1 references)
target     prot opt source               destination

Chain IN_public_allow (1 references)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere             udp dpts:8760:9960 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpts:8760:9960 ctstate NEW,UNTRACKED
ACCEPT     udp  --  anywhere             anywhere             udp dpts:18760:19960 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpts:18760:19960 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh ctstate NEW,UNTRACKED
ACCEPT     udp  --  anywhere             anywhere             udp dpt:ntp ctstate NEW,UNTRACKED
ACCEPT     udp  --  anywhere             anywhere             udp dpts:33434:33634 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:5201 ctstate NEW,UNTRACKED
ACCEPT     udp  --  anywhere             anywhere             udp dpt:5201 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:5001 ctstate NEW,UNTRACKED
ACCEPT     udp  --  anywhere             anywhere             udp dpt:5001 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:5000 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:5101 ctstate NEW,UNTRACKED
ACCEPT     udp  --  anywhere             anywhere             udp dpt:5000 ctstate NEW,UNTRACKED
ACCEPT     udp  --  anywhere             anywhere             udp dpt:5101 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpts:5890:5900 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:4823 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:861 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:862 ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http ctstate NEW,UNTRACKED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:https ctstate NEW,UNTRACKED

Chain FWDI_public (1 references)
target     prot opt source               destination
FWDI_public_log  all  --  anywhere             anywhere
FWDI_public_deny  all  --  anywhere             anywhere
FWDI_public_allow  all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere

Chain FWDI_public_log (1 references)
target     prot opt source               destination

Chain FWDI_public_deny (1 references)
target     prot opt source               destination

Chain FWDI_public_allow (1 references)
target     prot opt source               destination

Chain FWDO_public (1 references)
target     prot opt source               destination
FWDO_public_log  all  --  anywhere             anywhere
FWDO_public_deny  all  --  anywhere             anywhere
FWDO_public_allow  all  --  anywhere             anywhere

Chain FWDO_public_log (1 references)
target     prot opt source               destination

Chain FWDO_public_deny (1 references)
target     prot opt source               destination

Chain FWDO_public_allow (1 references)
target     prot opt source               destination
root@psmall-poz1:/etc/apt#
laeti-tia commented 5 months ago

That is indeed looking good. I think we can just provide the updated iptables 1.8.5 package in our repository, copied from buster-backports so that users don't need to configure this repository themselves. See https://github.com/perfsonar/minor-packages/commit/bdc32d73314e5db61b74fb2f919420eb0d4a2bef

szymontrocha commented 5 months ago

I used a fresh Debian 10 and added this apt sources file: https://perfsonar-repo.geant.org/debian/perfsonar-5.0-snapshot.list. Then installed perfsonar-toolkit. Unfortunately it doesn't seem to set iptables rules. Even after running apt-get reinstall perfsonar-toolkit-security iptables are still empty

# iptables -V
iptables v1.8.2 (nf_tables)

apt-get upgrade writes: The following packages heave been kept back: iptables

laeti-tia commented 5 months ago

Well, that seems more complicated than anticipated. The new iptables version has a lot of dependencies coming from the buster-backports repository (see all packages ending in ~bpo10+ and then the dependencies to those packages) some of which might conflict with other things. I overlooked that and I actually think it's not a good idea to provide all those additional packages in our own repository.

I'd suggest that instead we write a FAQ entry stating that if users want to make full use of the perfsonar-toolkit-security package on Debian 10, they'll need to activate the Debian 10 backports repository. Or use the alternative approach mentioned by @igarny in https://github.com/perfsonar/toolkit/issues/427#issuecomment-1350770094

mfeit-internet2 commented 5 months ago

I'd suggest that instead we write a FAQ entry stating that if users want to make full use of the perfsonar-toolkit-security package on Debian 10, they'll need to activate the Debian 10 backports repository.

Would it be worth making activating that repository a standard step in the installation just as we do for EPEL on Red Hat systems?

laeti-tia commented 5 months ago

Would it be worth making activating that repository a standard step in the installation just as we do for EPEL on Red Hat systems?

I'm not sure it's needed as a general rule, but maybe. This is the only case where that would help, but maybe we could benefit from newer versions of some other packages too.

laeti-tia commented 4 months ago

@szymontrocha Here is what I suggest to add to the FAQ:

The perfsonar-toolkit-security package, which configures the firewall for perfSONAR purposes, is not being set up properly under a plain Debian 10 installation. This is due to an old iptables version provided with Debian 10. Using the updated packages from Debian 10 backports solves this issue and properly configures the firewall. To use this correction, you can follow the 3 simple steps described here:

  1. Add the following line in the /etc/apt/sources.list: deb http://deb.debian.org/debian buster-backports main

  2. Then refresh apt and install the new version with: apt-get update; apt-get install -t buster-backports iptables

  3. And finally re-install the perfsonar package to make sure everything is setup properly: apt-get reinstall perfsonar-toolkit-security

szymontrocha commented 3 months ago

I added a FAQ entry