Open seruff84 opened 4 years ago
Hi,
Do you replay/read pcaps or is it just running live? Do you have a reproducible case or pcap to share privately maybe ? It would be of great help.
Hi. it running live. Unfortunately company policy will not allow me to share pcap.
How often does it happen?
Can you share the output of
suricata --build-info
please?
from several minutes to several hours.
This is Suricata version 6.0.0-dev (1639dfa36 2020-07-28)
Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS TLS_C11 MAGIC RUST
SIMD support: none
Atomic intrinsics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 8.3.0, C version 201112
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.33, linked against LibHTP v0.5.33
Suricata Configuration:
AF_PACKET support: yes
eBPF support: no
XDP support: no
PF_RING support: no
NFQueue support: yes
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no
WinDivert enabled: no
Unix socket enabled: yes
Detection enabled: yes
Libmagic support: yes
libnss support: yes
libnspr support: yes
libjansson support: yes
hiredis support: yes
hiredis async with libevent: no
Prelude support: no
PCRE jit: yes
LUA support: yes, through luajit
libluajit: yes
GeoIP2 support: yes
Non-bundled htp: yes
Old barnyard2 support:
Hyperscan support: yes
Libnet support: yes
liblz4 support: yes
Rust support: yes
Rust strict mode: no
Rust compiler path: /root/.cargo/bin/rustc
Rust compiler version: rustc 1.45.0 (5c1f21c3b 2020-07-13)
Cargo path: /root/.cargo/bin/cargo
Cargo version: cargo 1.45.0 (744bd1fbb 2020-06-15)
Cargo vendor: yes
Python support: yes
Python path: /usr/bin/python3
Python distutils yes
Python yaml yes
Install suricatactl: yes
Install suricatasc: yes
Install suricata-update: yes
Profiling enabled: no
Profiling locks enabled: no
Development settings:
Coccinelle / spatch: yes
Unit tests enabled: no
Debug output enabled: no
Debug validation enabled: no
Generic build parameters:
Installation prefix: /usr
Configuration directory: /etc/suricata/
Log directory: /var/log/suricata/
--prefix /usr
--sysconfdir /etc
--localstatedir /var
--datarootdir /usr/share
Host: x86_64-pc-linux-gnu
Compiler: gcc (exec name) / g++ (real)
GCC Protect enabled: yes
GCC march native enabled: no
GCC Profile enabled: no
Position Independent Executable enabled: yes
CFLAGS -g -O2 -fdebug-prefix-map=/STAMUS/SELKS6/Suricata/suricata-2020072901=. -fstack-protector-strong -Wformat -Werror=format-security -std=c11 -I${srcdir}/../rust/gen
PCAP_CFLAGS -I/usr/include
SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security
What you can try if possible is (try/test first in QA setup) - is to compile with debug enabled: https://github.com/StamusNetworks/SELKS/wiki/How-to-compile-latest-Suricata-on-SELKS
then if there is a core extract the info like explained here https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Reporting_Bugs (gdb part)
ok - thank you for reporting that. I think there is a similar issue opened here - https://redmine.openinfosecfoundation.org/issues/3885
Hi! I had the same problem. The following change in the suricata.yaml file helped me: vlan: use-for-tracking: true -> use-for-tracking: false
thank. I'll try and wait for updates.
Hi! I had the same problem. The following change in the suricata.yaml file helped me: vlan: use-for-tracking: true -> use-for-tracking: false
It didn't work for me
Hi! I have compiled a new version from git, now it crashes with:
[3498] 26/8/2020 -- 09:54:04 - (source-af-packet.c:1784)
suricata --build info This is Suricata version 6.0.0-dev (d3cf2c21d 2020-08-25) Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS TLS_C11 MAGIC RUST SIMD support: none Atomic intrinsics: 1 2 4 8 byte(s) 64-bits, Little-endian architecture GCC version 8.3.0, C version 201112 compiled with _FORTIFY_SOURCE=2 L1 cache line size (CLS)=64 thread local storage method: _Thread_local compiled with LibHTP v0.5.33, linked against LibHTP v0.5.33
Suricata Configuration: AF_PACKET support: yes eBPF support: no XDP support: no PF_RING support: no NFQueue support: yes NFLOG support: no IPFW support: no Netmap support: no DAG enabled: no Napatech enabled: no WinDivert enabled: no
Unix socket enabled: yes Detection enabled: yes
Libmagic support: yes libnss support: yes libnspr support: yes libjansson support: yes hiredis support: yes hiredis async with libevent: no Prelude support: no PCRE jit: yes LUA support: yes, through luajit libluajit: yes GeoIP2 support: yes Non-bundled htp: yes Old barnyard2 support: Hyperscan support: yes Libnet support: yes liblz4 support: yes
Rust support: yes Rust strict mode: no Rust compiler path: /root/.cargo/bin/rustc Rust compiler version: rustc 1.45.2 (d3fb005a3 2020-07-31) Cargo path: /root/.cargo/bin/cargo Cargo version: cargo 1.45.1 (f242df6ed 2020-07-22) Cargo vendor: yes
Python support: yes Python path: /usr/bin/python3 Python distutils yes Python yaml yes Install suricatactl: yes Install suricatasc: yes Install suricata-update: yes
Profiling enabled: no Profiling locks enabled: no
Plugin support (experimental): yes
Development settings: Coccinelle / spatch: yes Unit tests enabled: no Debug output enabled: no Debug validation enabled: no
Generic build parameters: Installation prefix: /usr Configuration directory: /etc/suricata/ Log directory: /var/log/suricata/
--prefix /usr --sysconfdir /etc --localstatedir /var --datarootdir /usr/share
Host: x86_64-pc-linux-gnu Compiler: gcc (exec name) / g++ (real) GCC Protect enabled: yes GCC march native enabled: no GCC Profile enabled: no Position Independent Executable enabled: no CFLAGS -g -O2 -fdebug-prefix-map=/STAMUS/SELKS6/Suricata/suricata-2020072901=. -fstack-protector-strong -Wformat -Werror=format-security -std=c11 -I/../rust/gen -std=c11 -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist PCAP_CFLAGS -I/usr/include SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security
Seems a diff err than the first one reported above. A fix was merged into Suri today - so i will repackage and upload a package in the testing repo soon.
I builded from git today. This is Suricata version 6.0.0-dev (d3cf2c21d 2020-08-25) - Is this version already fixed?
Should contain the fix for https://redmine.openinfosecfoundation.org/issues/3885
Can you provide more info please about the core dump ? Like so https://github.com/StamusNetworks/SELKS/issues/248#issuecomment-671204317
The today's updates from git are in a package on the SELSK test repo. You could try them out. (you already have it build though - d3cf2c21d )
@seruff84 - if you have the info from the core - you could post it her or better of open a bug on the redmine with the full info from there?
If this help.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:265) <Info> (ConfYamlParse) -- Including configuration file /etc/suricata/selks6-addin.yaml.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'default-rule-path' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'rule-files' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'classification-file' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'reference-config-file' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'detect' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'default-log-dir' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'stats' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'outputs' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'logging' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'app-layer' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'asn1-max-frames' redefined.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:265) <Info> (ConfYamlParse) -- Including configuration file /etc/suricata/selks6-interfaces-config.yaml.
[7727] 26/8/2020 -- 14:12:01 - (conf-yaml-loader.c:289) <Info> (ConfYamlParse) -- Configuration node 'af-packet' redefined.
[7727] 26/8/2020 -- 14:12:01 - (suricata.c:1066) <Notice> (LogVersion) -- This is Suricata version 6.0.0-dev (d3cf2c21d 2020-08-25) running in SYSTEM mode
[7727] 26/8/2020 -- 14:12:01 - (util-cpu.c:178) <Info> (UtilCpuPrintSummary) -- CPUs/cores online: 10
[7727] 26/8/2020 -- 14:12:01 - (util-ioctl.c:112) <Info> (GetIfaceMTU) -- Found an MTU of 1500 for 'ens256'
[7727] 26/8/2020 -- 14:12:01 - (util-ioctl.c:112) <Info> (GetIfaceMTU) -- Found an MTU of 1500 for 'ens256'
[7727] 26/8/2020 -- 14:12:01 - (util-ioctl.c:112) <Info> (GetIfaceMTU) -- Found an MTU of 1500 for 'ens256'
[7727] 26/8/2020 -- 14:12:01 - (util-ioctl.c:112) <Info> (GetIfaceMTU) -- Found an MTU of 1500 for 'ens256'
[7727] 26/8/2020 -- 14:12:01 - (flow.c:635) <Notice> (FlowInitConfig) -- flow size 320, memcap allows for 419430 flows. Per hash row in perfect conditions 6
[7727] 26/8/2020 -- 14:12:01 - (util-logopenfile.c:571) <Info> (SCConfLogOpenGeneric) -- eve-log output device (regular) initialized: eve.json
[7727] 26/8/2020 -- 14:12:01 - (output-json-email-common.c:441) <Info> (OutputEmailInitConf) -- Going to log the md5 sum of email body
[7727] 26/8/2020 -- 14:12:01 - (output-json-email-common.c:445) <Info> (OutputEmailInitConf) -- Going to log the md5 sum of email subject
[7727] 26/8/2020 -- 14:12:01 - (output-json-dnp3.c:299) <Info> (OutputDNP3LogInitSub) -- DNP3 log sub-module initialized.
[7727] 26/8/2020 -- 14:12:01 - (output-json-dnp3.c:299) <Info> (OutputDNP3LogInitSub) -- DNP3 log sub-module initialized.
[7727] 26/8/2020 -- 14:12:01 - (log-pcap.c:1307) <Info> (PcapLogInitCtx) -- Using log dir /data/nsm/
[7727] 26/8/2020 -- 14:12:01 - (log-pcap.c:1418) <Info> (PcapLogInitCtx) -- Selected pcap-log compression method: none
[7727] 26/8/2020 -- 14:12:01 - (log-pcap.c:1422) <Info> (PcapLogInitCtx) -- using multi logging
[7727] 26/8/2020 -- 14:12:01 - (util-logopenfile.c:571) <Info> (SCConfLogOpenGeneric) -- stats output device (regular) initialized: stats.log
[7727] 26/8/2020 -- 14:12:01 - (util-conf.c:161) <Info> (ConfUnixSocketIsEnable) -- Running in live mode, activating unix socket
[7727] 26/8/2020 -- 14:12:01 - (reputation.c:635) <Info> (SRepInit) -- Loading reputation file: /etc/suricata/rules/scirius-iprep.list
[7727] 26/8/2020 -- 14:12:09 - (detect-engine-loader.c:355) <Info> (SigLoadSignatures) -- 1 rule files processed. 22913 rules successfully loaded, 0 rules failed
[7727] 26/8/2020 -- 14:12:09 - (util-threshold-config.c:1091) <Info> (SCThresholdConfParseFile) -- Threshold config parsed: 0 rule(s) found
[7727] 26/8/2020 -- 14:12:09 - (detect-engine-build.c:1416) <Info> (SigAddressPrepareStage1) -- 22918 signatures processed. 14 are IP-only rules, 3979 are inspecting packet payload, 18869 inspect application layer, 0 are decoder event only
[7727] 26/8/2020 -- 14:12:38 - (util-runmodes.c:264) <Info> (RunModeSetLiveCaptureWorkersForDevice) -- Going to use 10 thread(s)
[7728] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7728] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7729] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7729] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7730] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7730] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7731] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7731] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7732] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7732] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7733] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7733] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7734] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7734] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7735] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7735] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7736] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7736] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7737] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7737] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7727] 26/8/2020 -- 14:12:38 - (util-runmodes.c:264) <Info> (RunModeSetLiveCaptureWorkersForDevice) -- Going to use 10 thread(s)
[7738] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7738] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7739] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7739] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7740] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7740] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7741] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7741] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7742] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7742] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7743] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7743] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7744] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7744] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7745] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7745] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7746] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7746] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7747] 26/8/2020 -- 14:12:38 - (log-pcap.c:761) <Info> (PcapLogInitRingBuffer) -- Initializing PCAP ring buffer for /data/nsm//log.%n.%t.pcap.
[7747] 26/8/2020 -- 14:12:38 - (log-pcap.c:902) <Notice> (PcapLogInitRingBuffer) -- Ring buffer initialized with 1 files.
[7748] 26/8/2020 -- 14:12:38 - (flow-manager.c:806) <Notice> (FlowManager) -- FM FM#01/0 starting. min_timeout 30s. Full hash pass in 240s
[7727] 26/8/2020 -- 14:12:38 - (util-conf.c:161) <Info> (ConfUnixSocketIsEnable) -- Running in live mode, activating unix socket
[7727] 26/8/2020 -- 14:12:38 - (unix-manager.c:132) <Info> (UnixNew) -- Using unix socket file '/var/run/suricata/suricata-command.socket'
[7727] 26/8/2020 -- 14:12:38 - (tm-threads.c:1965) <Notice> (TmThreadWaitOnThreadInit) -- all 20 packet processing threads, 4 management threads initialized, engine started.
[7747] 26/8/2020 -- 14:12:39 - (source-af-packet.c:507) <Info> (AFPPeersListReachedInc) -- All AFP capture threads are running.
suricata: app-layer-parser.c:1264: AppLayerParserParse: Assertion `!(res.needed + res.consumed < input_len)' failed.
Aborted (core dumped)
Yep thank you! I have opened an issue here - https://redmine.openinfosecfoundation.org/issues/3896
suricata randomly stop capture package with suricata: stream-tcp-reassemble.c:1066: AdjustToAcked: Assertion `!(adjusted > check)' failed. Aborted (core dumped)