search

stamparm / maltrail

Malicious traffic detection system
MIT License
6.58k stars 1.09k forks source link

Has there been any change in the blacklist updates in the past week? #19272

Closed secdoc closed 3 months ago

secdoc commented 3 months ago

Question Has there been any change in the blacklist updates in the past week?

Support It seems that Maltrail on OPNsense was fine up to 8/7/2024 as far as over all visibility, but after that time it has progressively diminished to the point that there is no hits at all as of today...

8/7/2024 image 8/12/2024 image overall activity by date (maltrail calendar view) image

MikhailKasimov commented 3 months ago

Hello!

Nothing was changed. Please, check sensor's status, if it is run or no.

secdoc commented 3 months ago

@MikhailKasimov the client is running. image image image

stamparm commented 3 months ago

looks like sensor is dead (not web server). you should restart it somehow

secdoc commented 3 months ago

looks like sensor is dead (not web server). you should restart it somehow

It is showing as having been restarted, but I will restart the OPNsense FW to see if that makes a difference..., but from a process perspective all is showing as running and recreated when restarted....

MikhailKasimov commented 3 months ago

@secdoc Please, attach here all last maltrail's log from /var/log/maltrail, let's try in investigate the reason of sensor has got stopped. Because anyway it is abnormal. Thanks!

secdoc commented 3 months ago

@MikhailKasimov do you want "all" or the last/most resent set covering the last week or two?

MikhailKasimov commented 3 months ago

@MikhailKasimov do you want "all" or the last/most resent set covering the last week or two?

image

For these days, for example....

secdoc commented 3 months ago

Here are the logs.... maltrail.tar.gz

MikhailKasimov commented 3 months ago

Interesting, why SIGTERM...

image

@mimugmail Hello! Could you, please, give a hint how suchlike cases could be debugged in OPNSense? Thank you!

secdoc commented 3 months ago

Just an FYI I created a ticket in the OPNsense community discussion board

secdoc commented 3 months ago

Some additional data points and not sure if it would be a rabbit trail to nothing but an issue where ICMP is being sent to the loopback address from the loopback in high volumes?

Here is example logs"

<134>1 2024-08-14T11:56:43+00:00 xxxx.acme.tech filterlog 12611 - [meta sequenceId="7416432"] 66,,,1232f88e5fac29a32501e3f051020cac,lo0,match,pass,out,4,0x0,,64,1281,0,none,1,icmp,596,127.0.0.1,127.0.0.1,datalength=576

When looking at the logs there is a rather large volume of traffic to the loopback from the loopback which does not make sense.

2024-08-14_07-37

MikhailKasimov commented 3 months ago

Please, attach also ps -aux and netstat commands outputs... Because all this "activity" seems not to be a normal one.

secdoc commented 3 months ago

Please, attach also ps -aux and netstat commands outputs... Because all this "activity" seems not to be a normal one.

Agree... see attached ps_aux.txt netstat.txt

SgtDornan8 commented 3 months ago

Today I discovered oddities in the behavior of MT 0.66 on OPNsense. The databases have not been updated since January 31, 2024. Now I am conducting an experiment with manually updating the MT version to 0.72. I'm waiting until Monday to see if the automatic update will work image

secdoc commented 3 months ago

For reference, this is the spec's of the OPNsense system:

Versions: OPNsense 24.7.1-amd64 FreeBSD 14.1-RELEASE-p3 OpenSSL 3.0.14 CPU: 12th Gen Intel(R) Core(TM) i5-1240P Memory: 32GB

Installed Modules: os-acme-client os-clamav os-crowdsec os-dmidecode os-etpro-telemetry os-git-backup os-haproxy
os-intrusion-detection-content-et-open os-intrusion-detection-content-et-pro
os-intrusion-detection-content-pt-open os-intrusion-detection-content-snort-vrt os-iperf os-maltrail os-sensei os-sensei-agent os-sensei-updater os-sunnyvalley os-theme-cicada os-theme-rebellion os-vnstat

MikhailKasimov commented 3 months ago

@secdoc Honestly speaking, currently I have no idea what is this bedlam at all... Is it an option to plan a conf-call (Meet, Teams, Zoom - doesn't matter) on Saturday, for example?

mimugmail commented 3 months ago

Currently off for two weeks, cant have a look

secdoc commented 3 months ago

Weekend would be fine and can do zoom to discuss... Just need Timezone...

MikhailKasimov commented 3 months ago

Weekend would be fine and can do zoom to discuss... Just need Timezone...

EEST

secdoc commented 3 months ago

How does 6PM EEST or 10 AM CST work for you on Sat 8/17?

MikhailKasimov commented 3 months ago

How does 6PM EEST or 10 AM CST work for you on Sat 8/17?

Yes, 6PM EEST on Sat 8/17 is fine.

secdoc commented 3 months ago

Do you want to do zoom or want me to setup?

MikhailKasimov commented 3 months ago

Do you want to do zoom or want me to setup?

Zoom. Wanna take a look by my eyes.

secdoc commented 3 months ago

@MikhailKasimov here is the invite:

Lester Nichols is inviting you to a scheduled Zoom meeting.

Topic: Lester Nichols' Zoom Meeting Time: Aug 17, 2024 10:00 Central Time (US and Canada)

Join Zoom Meeting https://us06web.zoom.us/j/86162175588?pwd=f8tKpgB9yq3XbNE8JLBVahSR50hZIY.1

Meeting ID: 861 6217 5588 Passcode: 181892

MikhailKasimov commented 3 months ago

Hello!

@secdoc Would be interesting how's going with with latest OPNSense: https://forum.opnsense.org/index.php?topic=42355.0 and is it going on today better after we had a conf-call ?

secdoc commented 3 months ago

Hello!

@secdoc Would be interesting how's going with with latest OPNSense: https://forum.opnsense.org/index.php?topic=42355.0 and is it going on today better after we had a conf-call ?

@MikhailKasimov I had checked this morning (0500 CST) and there were no updates, but after seeing your message, it looks like they have pushed the updates to the repo. I will update to the latest version and let you know. As far as current issues, they still are the same, so looks like outbound (sourced from my network, primarily DNS lookups) shows but still no change in what should be expected volume.

secdoc commented 3 months ago

Hello! @secdoc Would be interesting how's going with with latest OPNSense: https://forum.opnsense.org/index.php?topic=42355.0 and is it going on today better after we had a conf-call ?

@MikhailKasimov I had checked this morning (0500 CST) and there were no updates, but after seeing your message, it looks like they have pushed the updates to the repo. I will update to the latest version and let you know. As far as current issues, they still are the same, so looks like outbound (sourced from my network, primarily DNS lookups) shows but still no change in what should be expected volume.

@MikhailKasimov I applied the update and there has been no change in traffic visibility. So it appears based on visibility, traffic patterns have drastically change on inbound activity in comparison to what what happening 2 weeks ago.

image

secdoc commented 3 months ago

I am closing the issue because I think this is more associated now with a drastic traffic pattern change at the ingress. The errors or ancillary errors I think were red herrings...

MikhailKasimov commented 3 months ago

Ok on closing. Anyway, we've tried all variants to debug Maltrail's detection behavior due to conf-call session and detection was correct.

secdoc commented 2 months ago

This is just an update to this issue. After applying the OPNsense 24.7.3 update today, the Maltrail traffic patterns that were expected returned. So not sure if part of the issue was associated with the delta and/or changes between 24.7.2, but now seeing the visibility expected...

image

MikhailKasimov commented 2 months ago

This is just one more confirmation, that Maltrail works OK itself. The deal was in its environment. Thank you for letting know!