opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.34k stars 749 forks source link

20.7 devel suricata issues #4095

Closed L1ghtn1ng closed 3 years ago

L1ghtn1ng commented 4 years ago

On my test VM(in virtualbox) of Opnsense 20.7(Using the devel iso) fully up to date it does not block anything in IPS mode.

Using the test rule that has eicar in it does not get blocked and no alerts. This is not an issue on 20.1

The following is from what I have seen in the logs


2020-05-08T21:51:23 | suricata[54377]: [100187] <Notice> -- rule reload complete
-- | --
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ET.bit.do.shortener' is checked but not set. Checked in 2029550 and 0  other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ET.armwget' is checked but not set. Checked in 2024241 and 0 other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ET.Suspicious.Domain.Fake.Browser' is checked but not set. Checked in  2018572 and 0 other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'et.http.PK' is checked but not set. Checked in 2019835 and 3 other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ET.JavaNotJar' is checked but not set. Checked in 2016540 and 0 other  sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ET.Hancitor' is checked but not set. Checked in 2024605 and 0 other  sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'HTTP.UncompressedFlash' is checked but not set. Checked in 2018428 and 1  other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ETPRO.RTF' is checked but not set. Checked in 2020700 and 0 other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'exe.no.referer' is checked but not set. Checked in 2020500 and 0 other  sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'et.WinHttpRequest' is checked but not set. Checked in 2019823 and 0  other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ET.http.javaclient.vulnerable' is checked but not set. Checked in  2015849 and 0 other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ET.pdf.in.http' is checked but not set. Checked in 2017790 and 1 other  sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'et.JavaArchiveOrClass' is checked but not set. Checked in 2017768 and  11 other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'ET.http.javaclient' is checked but not set. Checked in 2017909 and 11  other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit  'et.IE7.NoRef.NoCookie' is checked but not set. Checked in 2024192 and 1  other sigs
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] -  signature at  /usr/local/etc/suricata/opnsense.rules/emerging-phishing.rules:1720 uses  unknown classtype: "domain-c2", using default priority 3. This message  won't be shown again for this classtype
2020-05-08T21:50:56 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] -  signature at  /usr/local/etc/suricata/opnsense.rules/emerging-phishing.rules:72 uses  unknown classtype: "credential-theft", using default priority 3. This  message won't be shown again for this classtype
2020-05-08T21:50:55 | suricata[54377]:  [100187] <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] -  error parsing signature "drop  tls $EXTERNAL_NET any -> $HOME_NET any  (msg:"ET JA3 Hash - Suspected Cobalt Strike Malleable C2 (ja3s) M1";  flow:established,from_server; ja3s.hash;  content:"649d6810e8392f63dc311eecb6b7098b"; tls.cert_subject;  content:!"servicebus.windows.net"; flowbits:isset,ET.cobaltstrike.ja3;  metadata: former_category JA3; classtype:command-and-control;  sid:2028832; rev:1; metadata:affected_product  Windows_XP_Vista_7_8_10_Server_32_64_Bit, attack_target Client_Endpoint,  deployment Perimeter, signature_severity Major, created_at 2019_10_15,  malware_family Cobalt_Strike, updated_at 2019_10_15;)" from file  /usr/local/etc/suricata/opnsense.rules/emerging-ja3.rules at line 44
2020-05-08T21:50:55 | suricata[54377]: [100187] <Error> -- [ERRCODE: SC_WARN_JA3_DISABLED(309)] - ja3(s) support is not enabled
2020-05-08T21:50:55 | suricata[54377]:  [100187] <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] -  error parsing signature "alert tls $HOME_NET any -> $EXTERNAL_NET any  (msg:"ET JA3 Hash - Suspected Cobalt Strike Malleable C2 M1 (set)";  flow:established,to_server; ja3.hash;  content:"eb88d0b3e1961a0562f006e5ce2a0b87"; ja3.string;  content:"771,49192-49191-49172-49171"; flowbits:set,ET.cobaltstrike.ja3;  flowbits:noalert; metadata: former_category JA3;  classtype:command-and-control; sid:2028831; rev:1;  metadata:affected_product Windows_XP_Vista_7_8_10_Server_32_64_Bit,  attack_target Client_Endpoint, deployment Perimeter, signature_severity  Major, created_at 2019_10_15, malware_family Cobalt_Strike, updated_at  2019_10_15;)" from file  /usr/local/etc/suricata/opnsense.rules/emerging-ja3.rules at line 42
2020-05-08T21:50:55 | suricata[54377]: [100187] <Error> -- [ERRCODE: SC_WARN_JA3_DISABLED(309)] - ja3 support is not enabled
2020-05-08T21:50:55 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] -  signature at  /usr/local/etc/suricata/opnsense.rules/emerging-hunting.rules:396 uses  unknown classtype: "external-ip-check", using default priority 3. This  message won't be shown again for this classtype
2020-05-08T21:50:55 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] -  signature at  /usr/local/etc/suricata/opnsense.rules/emerging-exploit_kit.rules:1464  uses unknown classtype: "social-engineering", using default priority 3.  This message won't be shown again for this classtype
2020-05-08T21:50:55 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] -  signature at  /usr/local/etc/suricata/opnsense.rules/emerging-exploit_kit.rules:828  uses unknown classtype: "targeted-activity", using default priority 3.  This message won't be shown again for this classtype
2020-05-08T21:50:55 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] -  signature at  /usr/local/etc/suricata/opnsense.rules/emerging-exploit.rules:796 uses  unknown classtype: "command-and-control", using default priority 3. This  message won't be shown again for this classtype
2020-05-08T21:50:55 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] -  signature at  /usr/local/etc/suricata/opnsense.rules/emerging-exploit.rules:746 uses  unknown classtype: "exploit-kit", using default priority 3. This message  won't be shown again for this classtype
2020-05-08T21:50:55 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_UNKNOWN_VALUE(129)] -  signature at  /usr/local/etc/suricata/opnsense.rules/emerging-coinminer.rules:42 uses  unknown classtype: "coin-mining", using default priority 3. This message  won't be shown again for this classtype
2020-05-08T21:50:49 | suricata[54377]: [100187] <Notice> -- rule reload starting
2020-05-08T21:43:10 | suricata[54377]: [100187] <Notice> -- all 2 packet processing threads, 4 management threads initialized, engine started.
2020-05-08T21:43:10 | suricata[54377]: [100224] <Notice> -- opened netmap:em1/T from em1: 0x4a5719fc300
2020-05-08T21:43:09 | suricata[54377]: [100224] <Notice> -- opened netmap:em1^ from em1^: 0x4a5719fc000
2020-05-08T21:43:07 | suricata[54377]: [100216] <Notice> -- opened netmap:em1^ from em1^: 0x4a55c8b2300
2020-05-08T21:43:06 | suricata[54377]: [100216] <Notice> -- opened netmap:em1/R from em1: 0x4a55c8b2000
2020-05-08T21:43:05 | suricata[54377]:  [100187] <Warning> -- [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 1  rule files specified, but no rule was loaded at all!
2020-05-08T21:43:05 | suricata[26541]:  [100140] <Warning> -- [ERRCODE: SC_WARN_NO_JA3_SUPPORT(308)] - no  MD5 calculation support built in (LibNSS), disabling JA3
2020-05-08T21:43:05 | suricata[26541]: [100140] <Notice> -- This is Suricata version 5.0.3 RELEASE running in SYSTEM mode

There is also issues with ja3 support not being enabled due to issues with LibNSS not having MD5 support

AdSchellevis commented 4 years ago

I need to check the ja3 stuff, but have you updated to the latests 20.7 version? (released yesterday if I'm not mistaken)

L1ghtn1ng commented 4 years ago

Yes as I have the new php version 7.3.14 and 5.0.3 of suricata etc


From: Ad Schellevis notifications@github.com Sent: Saturday, May 9, 2020 8:58:12 AM To: opnsense/core core@noreply.github.com Cc: J.Townsend townsend891@hotmail.com; Author author@noreply.github.com Subject: Re: [opnsense/core] 20.7 devel suricata issues (#4095)

I need to check the ja3 stuff, but have you updated to the latests 20.7 version? (released yesterday if I'm not mistaken)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/opnsense/core/issues/4095#issuecomment-626125087, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AA3V2QX6VST2HIXWPNK6SBTRQUEJJANCNFSM4M4QIXUQ.

AdSchellevis commented 4 years ago

Just tried the same on a test machine over here, seems to be working like a charm:


# curl -o /dev/null http://pkg.opnsense.org/test/eicar.com.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0

image

ja3 support isn't enabled on our suricata version, but we probably should consider that for 20.7, I expect the same messages on 20.1 by the way it wasn't enabled there either

L1ghtn1ng commented 4 years ago

Okay will try what you did and yeah would be good to ja3 support enabled


From: Ad Schellevis notifications@github.com Sent: Saturday, May 9, 2020 2:40:08 PM To: opnsense/core core@noreply.github.com Cc: J.Townsend townsend891@hotmail.com; Author author@noreply.github.com Subject: Re: [opnsense/core] 20.7 devel suricata issues (#4095)

Just tried the same on a test machine over here, seems to be working like a charm:

curl -o /dev/null http://pkg.opnsense.org/test/eicar.com.txt

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0

[image]https://user-images.githubusercontent.com/9783985/81475162-ba3b5400-920a-11ea-8800-77c519233c39.png

ja3 support isn't enabled on our suricata version, but we probably should consider that for 20.7, I expect the same messages on 20.1 by the way it wasn't enabled there either

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/opnsense/core/issues/4095#issuecomment-626177835, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AA3V2QQH2NN5G4K6Z5CTI4TRQVMLRANCNFSM4M4QIXUQ.

L1ghtn1ng commented 4 years ago

So doing your command works, but if doing it from https://www.eicar.org/?page_id=3950 does not when it use to before they updated the site/page

L1ghtn1ng commented 4 years ago

You are also only getting 1 alert in your screenshot where as I get 2 for 1 instance of running your command. This issue is not new but can there be something done about this?

image

mimugmail commented 4 years ago

Do you download via HTTPS?

J.Townsend notifications@github.com schrieb am Sa., 9. Mai 2020, 17:02:

You are also only getting 1 alert in your screenshot where as I get 2 for 1 instance of running your command. This issue is not new but can there be something done about this?

[image: image] https://user-images.githubusercontent.com/3628354/81477207-6c284f80-920e-11ea-984a-0b14fc5a0cc7.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/opnsense/core/issues/4095#issuecomment-626189036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE355IMXYFDBTLG6WLPNF73RQVWAFANCNFSM4M4QIXUQ .

L1ghtn1ng commented 4 years ago

Nope http

L1ghtn1ng commented 4 years ago

The download area has HTTP and HTTPS sections even though the main page is HTTPS I picked the HTTP

AdSchellevis commented 4 years ago

maybe it redirects to https, either way it's not something we can fix. the duplicate messages are a result of drop/alert log if I'm not mistaken, also doesn't seem to be related to our configuration.

L1ghtn1ng commented 4 years ago

I have just noticed that the interface is not showing in the screenshot and have also just confirmed that with the standard theme as well

lordraiden commented 4 years ago

Just tried the same on a test machine over here, seems to be working like a charm:


# curl -o /dev/null http://pkg.opnsense.org/test/eicar.com.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0

image

ja3 support isn't enabled on our suricata version, but we probably should consider that for 20.7, I expect the same messages on 20.1 by the way it wasn't enabled there either

Could you confirm if ja3 will be available in 20.7 or is too early to know? thanks

AdSchellevis commented 4 years ago

@lordraiden it's scheduled, yes https://github.com/opnsense/tools/commit/cc731dcc812d19758e1db4ce93e0f02b793e99a9 . Although this obviously doesn't make a difference for the https/eicar question (the rule needs content to process).

AdSchellevis commented 3 years ago

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.