pavel-odintsov / fastnetmon

FastNetMon - very fast DDoS sensor with sFlow/Netflow/IPFIX/SPAN support
https://fastnetmon.com
GNU General Public License v2.0
3.39k stars 561 forks source link

Issues with package from Debian Bookworm backprots #998

Open przemeksw opened 7 months ago

przemeksw commented 7 months ago

Version: 1.2.6 System - debian 12.4

Hi. After starting fastnetmon using "service fastnetmon start" I wait for a very long time. journalctl -xe - says that pid cannot be created in the specified path - even though the access rights are there, because fastnetmon is run as root. Please let me know how to improve it

root@ddos:/run# service fastnetmon status × fastnetmon.service - FastNetMon - DoS/DDoS analyzer with sFlow/Netflow/mirror support Loaded: loaded (/lib/systemd/system/fastnetmon.service; enabled; preset: enabled) Active: failed (Result: timeout) since Sun 2024-01-07 22:53:50 CET; 1min 32s ago Duration: 9.789s Docs: man:fastnetmon(8) Process: 3015 ExecStart=/usr/sbin/fastnetmon --daemonize (code=exited, status=0/SUCCESS) CPU: 1.852s

Jan 07 22:52:20 ddos systemd[1]: Starting fastnetmon.service - FastNetMon - DoS/DDoS analyzer with sFlow/Netflow/mirror support... Jan 07 22:52:20 ddos fastnetmon[3015]: We will run in daemonized mode Jan 07 22:52:20 ddos systemd[1]: fastnetmon.service: Can't open PID file /run/fastnetmon.pid (yet?) after start: No such file or directory Jan 07 22:53:50 ddos systemd[1]: fastnetmon.service: start operation timed out. Terminating. Jan 07 22:53:50 ddos systemd[1]: fastnetmon.service: Failed with result 'timeout'. Jan 07 22:53:50 ddos systemd[1]: Failed to start fastnetmon.service - FastNetMon - DoS/DDoS analyzer with sFlow/Netflow/mirror support. Jan 07 22:53:51 ddos systemd[1]: fastnetmon.service: Consumed 1.852s CPU time.

pavel-odintsov commented 7 months ago

Hello!

This error is not critical can does not mean that there are any issues.

Please try running FastNetMon daemon as systems runs it and check for possible error during it.

I can recommend reverting configuration to original one as crash may be cause by something in configuration.

Also worth checking dmesg for potential issues and look for anything in /var/log/fastnetmon.log

If you accidentally added 0.0.0.0/0 in networks list or added network large then /16 it will crash due to lack of memory too.

Sincerely yours, Pavel Odintsov

On Sun, 7 Jan 2024 at 21:58, przemeksw @.***> wrote:

Version: 1.2.6 System - debian 12.4

Hi. After starting fastnetmon using "service fastnetmon start" I wait for a very long time. journalctl -xe - says that pid cannot be created in the specified path - even though the access rights are there, because fastnetmon is run as root. Please let me know how to improve it

@.***:/run# service fastnetmon status × fastnetmon.service - FastNetMon - DoS/DDoS analyzer with sFlow/Netflow/mirror support Loaded: loaded (/lib/systemd/system/fastnetmon.service; enabled; preset: enabled) Active: failed (Result: timeout) since Sun 2024-01-07 22:53:50 CET; 1min 32s ago Duration: 9.789s Docs: man:fastnetmon(8) Process: 3015 ExecStart=/usr/sbin/fastnetmon --daemonize (code=exited, status=0/SUCCESS) CPU: 1.852s

Jan 07 22:52:20 ddos systemd[1]: Starting fastnetmon.service - FastNetMon

  • DoS/DDoS analyzer with sFlow/Netflow/mirror support... Jan 07 22:52:20 ddos fastnetmon[3015]: We will run in daemonized mode Jan 07 22:52:20 ddos systemd[1]: fastnetmon.service: Can't open PID file /run/fastnetmon.pid (yet?) after start: No such file or directory Jan 07 22:53:50 ddos systemd[1]: fastnetmon.service: start operation timed out. Terminating. Jan 07 22:53:50 ddos systemd[1]: fastnetmon.service: Failed with result 'timeout'. Jan 07 22:53:50 ddos systemd[1]: Failed to start fastnetmon.service - FastNetMon - DoS/DDoS analyzer with sFlow/Netflow/mirror support. Jan 07 22:53:51 ddos systemd[1]: fastnetmon.service: Consumed 1.852s CPU time.

— Reply to this email directly, view it on GitHub https://github.com/pavel-odintsov/fastnetmon/issues/998, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU56ZVWZGUH3JGJJGN3ZXLYNMLA7AVCNFSM6AAAAABBQTQCN6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3DSMZQG4YTQMY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

przemeksw commented 7 months ago

I have a /20 monitoring network - so it's not that big. The configuration file is original - without any modifications. The process itself is starting - as you write - but I prefer it to be correct

przemeksw commented 7 months ago

Jan 07 23:11:38 ddos fastnetmon[3332]: We will run in daemonized mode Jan 07 23:11:38 ddos systemd[1]: fastnetmon.service: Can't open PID file /run/fastnetmon.pid (yet?) after start: No such file or directory Jan 07 23:13:08 ddos systemd[1]: fastnetmon.service: start operation timed out. Terminating. Jan 07 23:13:08 ddos systemd[1]: fastnetmon.service: Failed with result 'timeout'. ░░ Subject: Unit failed ░░ Defined-By: systemd ░░ Support: https://www.debian.org/support ░░ ░░ The unit fastnetmon.service has entered the 'failed' state with result 'timeout'. Jan 07 23:13:08 ddos systemd[1]: Failed to start fastnetmon.service - FastNetMon - DoS/DDoS analyzer with sFlow/Netflow/mirror support. ░░ Subject: A start job for unit fastnetmon.service has failed

pavel-odintsov commented 7 months ago

Hello!

You should not see timeouts, that's clearly means that there are some issues but pid error is just red herring.

If configuration is standard then it's clearly something wrong with machine itself.

/20 is perfectly fine.

Do you see messages in log?

Sincerely yours, Pavel Odintsov

On Sun, 7 Jan 2024 at 22:09, przemeksw @.***> wrote:

I have a /20 monitoring network - so it's not that big. The configuration file is original - without any modifications. The process itself is starting - as you write - but I prefer it to be correct

— Reply to this email directly, view it on GitHub https://github.com/pavel-odintsov/fastnetmon/issues/998#issuecomment-1880191373, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU56ZRONM6KP6FS3AB46V3YNMMKJAVCNFSM6AAAAABBQTQCN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBQGE4TCMZXGM . You are receiving this because you commented.Message ID: @.***>

przemeksw commented 7 months ago

2024-01-07 22:58:51,538 [ERROR] Could not create pid file, please check permissions: /run/fastneton/fastnetmon.pid 2024-01-07 22:59:00,449 [INFO] Logger initialized! 2024-01-07 22:59:00,450 [INFO] ExaBGP support initialized correctly 2024-01-07 22:59:00,450 [INFO] We read global ban settings: Configuration params: We call ban script: yes We call ban script for IPv6: no Packets per second: 29000 Mbps per second: 1100 Flows per second: 20000

2024-01-07 22:59:00,450 [ERROR] Could not create pid file, please check permissions: /run/fastneton/fastnetmon.pid 2024-01-07 22:59:09,823 [INFO] Logger initialized! 2024-01-07 22:59:09,824 [INFO] ExaBGP support initialized correctly 2024-01-07 22:59:09,824 [INFO] We read global ban settings: Configuration params:

pavel-odintsov commented 7 months ago

Hello!

Nothing unusual actually. I can recommend running it manually not from systemd and it may show some error on console.

We routinely test FastNetMon in Debian 12 and it definitely works: https://app.circleci.com/pipelines/github/pavel-odintsov/fastnetmon-community-packages?branch=master

If it does not then it's something in this machine and I do recommend deploying fresh VM or container and trying it again.

Sincerely yours, Pavel Odintsov

On Sun, 7 Jan 2024 at 22:16, przemeksw @.***> wrote:

2024-01-07 22:58:51,538 [ERROR] Could not create pid file, please check permissions: /run/fastneton/fastnetmon.pid 2024-01-07 22:59:00,449 [INFO] Logger initialized! 2024-01-07 22:59:00,450 [INFO] ExaBGP support initialized correctly 2024-01-07 22:59:00,450 [INFO] We read global ban settings: Configuration params: We call ban script: yes We call ban script for IPv6: no Packets per second: 29000 Mbps per second: 1100 Flows per second: 20000

2024-01-07 22:59:00,450 [ERROR] Could not create pid file, please check permissions: /run/fastneton/fastnetmon.pid 2024-01-07 22:59:09,823 [INFO] Logger initialized! 2024-01-07 22:59:09,824 [INFO] ExaBGP support initialized correctly 2024-01-07 22:59:09,824 [INFO] We read global ban settings: Configuration params:

— Reply to this email directly, view it on GitHub https://github.com/pavel-odintsov/fastnetmon/issues/998#issuecomment-1880194195, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU56ZSFOD52QUTUZJWELPTYNMNDXAVCNFSM6AAAAABBQTQCN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBQGE4TIMJZGU . You are receiving this because you commented.Message ID: @.***>

przemeksw commented 7 months ago

the problem is that if pid is not created, systemd automatically restarts the daemon. - I started it manually and it works now, but it doesn't change the fact that something is wrong. I have no idea why it behaves this way. Other applications including exabgp create pids in this location without any problems.

pavel-odintsov commented 7 months ago

Hello!

We actually completely changed our systemd configuration in 1.2.6: https://github.com/pavel-odintsov/fastnetmon/blob/9ee31530e4f012ad33c9fd70850221a21107945e/src/scripts/build_any_package.pl#L57

Are you using official package from Debian? Is it from Bookworm or from backports?

If it does not work then I can recommend reaching maintainers using Debian bug tracker as we do not maintain it on our own.

I'll check with our maintainer about systemd configuration but it will be changed only in future Debian releases or in backports

Sincerely yours, Pavel Odintsov

On Sun, 7 Jan 2024 at 22:49, przemeksw @.***> wrote:

the problem is that if pid is not created, systemd automatically restarts the daemon. - I started it manually and it works now, but it doesn't change the fact that something is wrong. I have no idea why it behaves this way. Other applications including exabgp create pids in this location without any problems.

— Reply to this email directly, view it on GitHub https://github.com/pavel-odintsov/fastnetmon/issues/998#issuecomment-1880203795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU56ZTI2DRAUETW5XJICP3YNMQ63AVCNFSM6AAAAABBQTQCN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBQGIYDGNZZGU . You are receiving this because you commented.Message ID: @.***>

przemeksw commented 7 months ago

I use backports, the official Bookworm repo is the older version 1.2.4-2

pavel-odintsov commented 7 months ago

Thank you for bug report. It's clearly bug in upstream Sid / Bookworm backports package and I'll share this issue with our maintainer.

You can manually fix it by making these adjustments https://github.com/pavel-odintsov/fastnetmon/commit/bffdec710912d18abe15f439f354788e2f7792f4 for /lib/systemd/system/fastnetmon.service

Then you need to reload systemd configuration:

sudo systemctl daemon-reload

And then restart FastNetMon:

sudo systemctl restart fastnetmon 
pavel-odintsov commented 7 months ago

Reported to upstream.

przemeksw commented 7 months ago

yes bffdec7 solves the problem. Thank you !

pavel-odintsov commented 7 months ago

Awesome. Sorry about this issue. We upgraded to 1.2.6 so easily that did not even check if everything is running smooth

pavel-odintsov commented 7 months ago

We added backports checking for CI and it immediately picked up issue.

pavel-odintsov commented 6 months ago

Issue was should be fixed in upstream.

pavel-odintsov commented 6 months ago

Still happening https://app.circleci.com/pipelines/github/pavel-odintsov/fastnetmon-community-packages/822/workflows/7a0cd7ec-a636-41de-98c6-2652be6da911/jobs/28821