search

rpthms / nft-geo-filter

Allow/deny traffic in nftables using country specific IP blocks
MIT License
97 stars 24 forks source link

Fallback provider (enhancement) #12

Closed canonex closed 3 years ago

canonex commented 3 years ago

Hi!

Could it be useful to have a fallback provider in case ipdeny.com is not reachable (for whatever reason -I saw the "dormant" when downloading- )?

In a script I started to write (not finished, I found this! ) I had used:

which I used exclusively in case the first one was not available. (often there are discrepancies between the IPs of the two services)

Thank you, Riccardo

rpthms commented 3 years ago

The "dormant" setting is to disable the geo-filter table when downloading the IP blocks because it's possible that the current set of rules in the geo-filter table could block access to ipdeny.com. It was easier to just always disable the rules while reaching out to ipdeny.com, rather then check if we have access to ipdeny.com or not and then take some action. So, just for a couple of seconds, the geo-filter table is set to dormant while the IP blocks get downloaded, after which the dormant status is removed.

A fallback provider is a good idea. We'll probably have to do some extra handling for ipverse.net IP blocks, because it looks like they have some comments on top of each country specific zone which needs to be ignored.

frankofno commented 3 years ago

Thanks for sharing ipverse.net as resource. There blocks look more optimized than those of ipdeny.com. Great idea to have a fallback. I would appreatiate that too. Maybe a check BEFORE flushing/overwriting the existing ipblocks when reaching out / updating to ipdeny.com? In case ipdeny.com is not available - do nothing or show message try later or something.

canonex commented 3 years ago

In case ipdeny.com is not available - do nothing or show message try later or something.

I think it is using the old zone set if there are problems with the new (line 356).

In fact I wrote to ipdeny that some address are wrong (in Germany): I can't use their set right now... (to cover this second problem a good add could be to have an argument to pass an alternative url like blabla.com/{}-addr.bla)

Other providers offers a similar listing, and others offers Creative Commons Share Alike db like maxmind, ip2location... But is a very different approach both on technical and freedom...

rpthms commented 3 years ago

@canonex @frankofno I've pushed a commit to add support for multiple providers using the --provider flag. Can you try it out? : https://github.com/rpthms/nft-geo-filter/blob/alternate_providers/nft-geo-filter

$ sudo ./nft-geo-filter --provider ipverse.net MC && sudo nft list ruleset
table inet geo-filter {
        set filter-v4 {
                type ipv4_addr
                flags interval
                auto-merge
                elements = { 37.44.224.0/22, 80.94.96.0/20,
                             82.113.0.0/19, 87.238.104.0/21,
                             87.254.224.0/19, 88.209.64.0/18,
                             91.199.109.0/24, 91.213.192.0/24,
                             176.114.96.0/20, 185.47.116.0/22,
                             185.162.120.0/22, 185.193.108.0/22,
                             185.250.4.0/22, 188.191.136.0/21,
                             193.34.228.0/23, 193.35.2.0/23,
                             194.9.12.0/23, 195.20.192.0/23,
                             195.78.0.0/19, 213.133.72.0/21 }
        }

        set filter-v6 {
                type ipv6_addr
                flags interval
                auto-merge
                elements = { 2a01:8fe0::/32,
                             2a06:92c0::/32,
                             2a07:9080::/29,
                             2a0b:8000::/29,
                             2a0f:b980::/29 }
        }

        chain filter-chain {
                type filter hook prerouting priority -190; policy accept;
                ip saddr @filter-v4 drop
                ip6 saddr @filter-v6 drop
        }
}
canonex commented 3 years ago

For me it ok, smooth as yesterday. I will make extensive use of exceptions to fill in the gaps in the list...

Thank you! Riccardo

./nft-geo-filter --log-drop --log-drop-prefix "GEOFILTER DROP " --allow --allow-established --log-accept --log-accept-level info --provider ipverse.net IT

~# nft list ruleset 
table ip filter {
    chain INPUT {
        type filter hook input priority 0; policy accept;
        meta l4proto tcp tcp dport { 80,443} counter packets 635833 bytes 89419041 jump f2b-apache-noscript
        meta l4proto tcp tcp dport 22 counter packets 5281 bytes 242732 jump f2b-ssh
        meta l4proto tcp tcp dport 22 counter packets 5281 bytes 242732 jump f2b-sshd
        meta l4proto tcp tcp dport { 80,443} counter packets 1166515 bytes 148488194 jump f2b-wordpress
    }

    chain FORWARD {
        type filter hook forward priority 0; policy accept;
    }

    chain OUTPUT {
        type filter hook output priority 0; policy accept;
    }

    chain f2b-wordpress {
        counter packets 1165826 bytes 148368992 return
    }

    chain f2b-sshd {
        counter packets 5281 bytes 242732 return
    }

    chain f2b-ssh {
        counter packets 5281 bytes 242732 return
    }

    chain f2b-apache-noscript {
        counter packets 635161 bytes 89374739 return
    }
}
table inet geo-filter {
    set filter-v4 {
        type ipv4_addr
        flags interval
        auto-merge
        elements = { 2.32.0.0/12, 2.56.92.0/22,
                 2.56.128.0/22, 2.57.32.0/22,
                 2.57.84.0/22, 2.58.136.0/22,
                 2.58.184.0/22, 2.58.208.0/22,
                        ...

                 217.196.128.0/20, 217.199.0.0/19,
                 217.200.0.0/14, 217.220.0.0/14 }
    }

    set filter-v6 {
        type ipv6_addr
        flags interval
        auto-merge
        elements = { 2001:678:12::/48,
                 2001:678:d4::/48,
                 2001:678:b14::/48,
                 2001:678:b50::/48,
                 2001:678:bd0::/48,
                 2001:678:c78::/48,
                 2001:67c:29c::/48,
                 2001:67c:790::/48,
                 2001:67c:1b08::/48,
                 2001:67c:21a0::/48,
                 2001:67c:2db0::/48,
                 2001:6b8::/32,
                        ...
                     2a10:e680::/29,
                 2a10:e700::/29,
                 2a10:eb00::/29,
                 2a10:f700::/29 }
    }

    chain filter-chain {
        type filter hook prerouting priority -190; policy drop;
        ct state established,related accept
        ip6 saddr { ::1, fe80::/10 } accept
        ip saddr { 10.0.0.0/8, 127.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 } accept
        ip saddr @filter-v4 log level info accept
        ip6 saddr @filter-v6 log level info accept
        log prefix "GEOFILTER DROP " drop
    }
}
rpthms commented 3 years ago

@canonex Thanks for the report. I've merged the pull request and added some documentation on the --provider flag to the README.

While my commits don't make the script fallback to another provider if one is unaccessible, you should easily be able to script that behaviour using something like:

# nft-geo-filter --allow --allow-established IT || nft-geo-filter --provider ipverse.net --allow --allow-established IT

That way, if nft-geo-filter errors out on ipdeny.com, it'll try to reach ipverse.net instead.

canonex commented 3 years ago

I do share here other resources I found, each with a different set of ips: http://www.iwik.org/ipcountry/