is this a bug with install ?

[cr0wm4n]

Hi,

Hope you can help, I am looking at moving to a purely nftables ruleset and came across this and thought i would give it a try :)

I am running ubuntu 22.04

I have used git clone to get the files and have configured it to install in /etc not /usr/local/etc

when running the install it does install into /etc fine but it is also installing the duplicate set of files into /etc/nftfw/etc_nftfw/

Is there a reason for the 2 copies of all the files and folders ?

Regards Mike

[pcollinson]

On Debian, where I have a package, the /etc/nftfw/etc_nftfw is a symlink to /usr/share/doc/nftfw/etc_nftfw. The doc directory also contains other stuff. The idea is that the install will not touch any files in /etc/nftfw on update, but the symlink allows for easy comparison of the working files with the new release.

So not a bug, a feature:-)

[cr0wm4n]

Super fast response thankyou ;) My install is not a sysmlink, so this location will be where it does the comparison ;)

Therefore I will just ignore that folder, and will configure and work with just the other files/folders under /etc/nftfw/ :)

Many thanks Mike

[cr0wm4n]

Another couple of questions not a bug :) when allowing ports, does it open both tcp and udp or just tcp ? if both how do i restrict to just tcp ? if just tcp, how do i open a udp port ?

Regards Mike

[pcollinson]

It opens both because that's what Symbiosis did. It uses the default accept.sh rule in rule.d which adds tcp and udp. The rules are called twice once for ipv4 and once for ipv6. The rules are simple shell scripts that echo stuff into the nft build stream. To tailor things, you can create a specific rule in rule.d. If your rule is called jim.sh, then you'd add DD-jim (where DD is the order) into the incoming.d directory. You'll have to explicitly add the port number into the rule. See ping.sh for an example. See the README in rule.d, the rule.d section in the User's Guide and 'man nftfw-files' for more on how these shell scripts work.

[cr0wm4n]

Thankyou for the suber fast reply again :) Much Appreciated

[cr0wm4n]

Sorry to be a pain, but i think i do have a real bug now :)

for Ubuntu, i dont think it has installed some of the files in the correct places.

man nftfw-files isnt finding the help file also i dont have the main scrpts in /usr/local/bin or in /bin i cannot find /usr/local/bin/nftfw :(

[pcollinson]

The README file is in /etc/nftfw/rule.d. If you've changed things to be installed with a root of /, then it will by default use: /etc/nftfw for control files, /usr/bin/ for the Python scripts - if you are doing that by hand, you'll need to run pip3 to install them, and /var/lib/nftfw for the 'private' files - this will be mostly empty - but needs to contain three directories: build.d, install.d and test.d. Hmm.. I suspect that pip3 will put the scripts in /usr/local/bin.

Manual pages may need hand installation.

[cr0wm4n]

Ok,

when i installed I only changed AUTO_DESTROOT from Y to N

this placed all the control files in /etc/nftfw as you suggested The folder /var/lib/nftfw is created with the 3 empty subfolders But There are no nftfw python scripts in any of /bin /usr/bin /usr/local/bin

also I am running this as root user so it isnt a permission issue and if it helps.. Python version installed is Python 3.10.4 pip version is pip 22.0.2

Any ideas how I can fix this please ?

Regards Mike

[pcollinson]

To install the scripts you need to run pip3 (or pip if it's working for Python 3).

The best plan is to look at the 'outdated' documents which are a step by step guide to installing by hand. You may have them in docs/outdated or you can find them at https://github.com/pcollinson/nftfw/blob/master/docs/outdated/Installation-Instructions.md (Manual installation instructions) which are step by step, or https://github.com/pcollinson/nftfw/blob/master/docs/outdated/Installation.md which is somewhat more verbose. The GitHub docs are formatted and probably easier to read than the raw md files.

These documents assume you are installing with a root of /usr/local - so you'll need to mentally edit the paths.

Do get back if these don't assist.

[cr0wm4n]

Thankyou very much again, these docs are more helpful :) I have confirmed all the required and suggested python/pip packages are installed but. this bit is not working :( $ sudo pip3 install . ... Successfully installed nftfw-

I am getting this error message

pip3 install ERROR: You must give at least one requirement to install (see "pip help install")

Regards Mike

[cr0wm4n]

ahhhh got them :) I had to simply add ./ to the pip3 install command like this pip3 install ./

it just told it im installing from the local directory :)

Sorry I have been so much of a pain hehe

[pcollinson]

Glad to assist.

[cr0wm4n]

as I dont like udp, In the default accept.sh i can just comment out the udp line and on the 1 port i do allow it i can create a seperate action for that port, this should be fine yes ?

[pcollinson]

That should do it. There are two helpful shell aliases that I use to look at nfables:

alias nfl='sudo nft list ruleset ip | less' alias nfl6='sudo nft list ruleset ip6 | less'

so you can easily see what's happening.

[cr0wm4n]

Really sorry to be a pain :) but I believe I have all the ports and services configured correct now. but i cannot find where to configure the interfaces themselves.

how do i tell it which is LAN eth0 and which is the WAN eth1 interface ? and how do i initialise it so LAN can be forwarded and use the internet

[pcollinson]

The distributed system supports a server machine. The basic template for the nftables rules in in /etc/nftfw/nftfw_init.nft. This can be replaced, look in the distribution for the nftfw_router_example directory. This contains the template I use on my internal gateway machine. My gateway is a router that is controlled by a web interface and I have that on a separate IP network so there are some rules that allow this.

[cr0wm4n]

Thankyou so much again, this is what i have had setup for many years on iptables.. my ubuntu box is also my gateway/router on the ubuntu box I do run several services locally (i know its not recommended ) including bind, apache and mail services

i have a ppp0 interface (wan) attached to eth0, then eth1 is the lan interface for my internal network

what i have currently done is export my iptables firewall and converted it to nfttables and loaded that via /etc/nftables.conf. it is very unclean and hard to follow but does still work as intended. I want to use your netfw as it looks like it will make my firewall so much easier to manage :)

[cr0wm4n]

Hi again, Besides the issues above, I think I have found a real bug :) I cannot create files with a | in them like 192.168.0.0|24 in ubuntu the | is a pipe, it pipes 1 command to another so when i run touch 192.168.0.0|24
it creates a file 192.168.0.0 and then tries to pipe the output to 24 and I get the error message 24: command not found

Regards Mike

[pcollinson]

You need to quote it: touch '192.168.0.0|24'

[cr0wm4n]

-rw-r--r-- 1 root root 0 Sep 1 17:59 '192.168.0.0|23'

it will be ok with the quotes when it looks for the files ?

[pcollinson]

That's a feature of the ls command on Linux... it's showing quotes - but they are not really there. You can prove that by echo 192* which will echo its arguments and say 192.168.0.0|24

[cr0wm4n]

unfortunately, i have not been able to get this correct, i have tried both of the template init.nft files changing the interfaces to match mine. but no matter what ive tried it just opens all ports on all interfaces. i have had to re-enable iptables my old and put the arno-iptables-firewall script back for now until i can get another box to try again :(

[pcollinson]

Hmm there is one possibility. I've had this problem recently. If you have been using iptables, then you must ensure that there are no active iptables in the kernel. Otherwise it will run both firewalls which can cause confusion. Running $ sudo iptables-legacy -F $ sudo ip6tables-legacy -F will ensure that they are flushed. Try $ sudo iptables-save to see you have nothing loaded.

You will have to use the router template - because you have two interfaces and will want to forward packets between them. I would expect the LAN interface to have all ports open.

Can you use the contact form on https://nftfw.uk/contact.php to send me your email address. I'll reply with my email address. Then you can send me your nftables.nft and I can see where you are with configuring this.

[cr0wm4n]

I just cracked it :))))

what i did I ended up dumping my existing ruleset and created my nftfw_init.nft based on that then with some modifications i was able to utilise the folders.

I still had the same problem, it was allowing everything on all cards.

I dumped the ruleset again and noticed 1 of the ports was very badly formed and it was basically just counter accept ARGGHHH

I knew at this point it wasnt the nftfw_init.nft so rechecked the tweaks i had made in the rule.d folder where i modified the default accept to only allow tcp and the file i created for dns on both udp and tcp.

it was this second file, it wasnt getting the parameters, it was a bad if else fi :)

I fixed that and am now up and running with nftfw :) Next I will try the other little apps and the webpage :)

Many thanks for all your help tho, I was pulling whats left of my hair out lol

[pcollinson]

Great... glad it's working for you.

[cr0wm4n]

im well happy finally getting it working :) i have since been trying the fail2ban integration I added the 2 actions to the action.d directory for some reason i havent yet figured out, it looks like the path in the action is not liked it is looking in /usr/bin not /usr/local/bin, my quick fix was just create a symlink in /usr/bin to /usr/local/bin/nftfwedit

I created the 2 nftfw banactions and commented out my existing nftables banactions and did test it, it does work, but was a lot slower than the nftables banactions at blocking ip's.

i then have renamed the nftfw banactions and uncommented the nftables banactions, it now looks like this.. banaction = nftables-multiport banaction_allports = nftables-allports banaction_mp = nftfw-multiport banaction_ap = nftfw-allports

I also created new actions

actionmp = %(action)s %(mta)s-whois-lines[sender="%(sender)s", dest="%(destemail)s", logpath="%(logpath)s", chain="%(chain)s"] %(banaction_mp)s[port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]

actionap = %(action)s %(mta)s-whois-lines[sender="%(sender)s", dest="%(destemail)s", logpath="%(logpath)s", chain="%(chain)s"] %(banaction_ap)s[port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]

now if i use action_ap or action_mp I get the ban itself created by fail2ban directly in nftables and i get my email notification with the whois and relevent logfile entries and i get the ban in nftfw as well.

it is working good, But I have one question In the nftfw-allports.conf and nftw-multiport.conf if i change the command parameters from

/usr/bin/nftfwedit -b to /usr/bin/nftfwedit -a

Would this have just the affect of just recording the ban and not actually setting a ban.

Reason I ask is I will continue to use fail2ban to perform the bans, for me it appears to be quicker at setting the ban but it would be nice to keep the record of the ban in nftfw :) would i also need to turn off log monitoring for nftfw so nftfw doesnt set the same ban as fail2ban ? but I do like the idea of keeping a record of them knocking on the door too

This what I am trying to achieve is

1. to keep nftfw as i love the way we can add the rules and lists using the files
2. the fastest possible speed of a ban being set when detected
3. a record of all bans set by both fail2ban or nftfw manual files
4. the knocking of the door record

Thanks again Mike

[pcollinson]

The -b option to nffwedit adds the ip to the database, and also creates a file in /etc/nftfw/blacklist.d, which will add the IP to the nftable rules when nftfw -f load is run. The -a flag omits this last step.

On Debian, with systemd, I've got a pair of systemd actions, one's a 'path' trips when any file changes in the /etc/ntftfw directories. It calls a service that runs nftfw -q load when it trips. I believe, but am not sure, that Ubuntu doesn't use systemd so you may be missing the ability to run a reload when something is added or removed from the blacklist.d directory. So using nftfwedit will not automatically trigger a reload of the tables. You might want to look for a replacement for the call program on file change feature. In the interim you might like to run the load function somewhat more frequently than is suggested in the distributed cron files.

I did some fail2ban actions for another user, and they were added into the distribution. I don't use it, if you are wanting to use fail2ban for detection, then you may get two hits from the same event - which may be OK. Ntffw will just increment the count in its database, so the IP may look 'badder' than it really is.

[cr0wm4n]

Thankyou again, ubuntu does use systemd so I have got this working too and removed my changes to fail2ban :)

On ubuntu we have rsyslog.d for logging do you on debian ? we also have journald

On ubuntu all kernel logging by default gets logged in /var/log/kern.log

this can be further broken down by types info warning error etc into seperate files. like this /var/log/kern.inf /var/log/kern.err /var/log/kern.warn etc

these nftables logs are kernel type info lines Sep 4 09:01:03 mail kernel: [298989.844206] Blacklist IN=ppp0 OUT= MAC= SRC=5.34

now i have enabled the door knocking the only way to make it work is to allow these kernel type info messages go into /var/log/syslog as well doing this i lose visibilty of all other system logging, it quickly gets buried and as there is a lot of door knocking these logfiles get big quick :)

is there a config option that will allow it to read from the /var/log/kern.* logfiles instead for the door knocking messages

[pcollinson]

Yes the file that the patterns look at is in the pattern file in patterns.d and just needs editing.

[cr0wm4n]

thats awesome, thankyou much :) I have spent ages looking through the code to see if i could find anyway lol