crashed when high traffic：doNormal16 (mode=CALLBACK_OUTPUT, do_accel=0 '\000', s=1,

[worldpeace365]

The hs was used in suricata， when the http traffic up to 2Gbps it crashed with only one thread，or less traffic with 4 threads。the core dump is follow：

Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `./suricata --runmode workers -Q 4 -c suricata.yaml --set mpm-algo=hs'. Program terminated with signal 11, Segmentation fault.

0 doNormal16 (mode=CALLBACK_OUTPUT, do_accel=0 '\000', s=1,

end=0x7fa59872864e <Address 0x7fa59872864e out of bounds>, c_inout=<synthetic pointer>, m=0x126eb600)
at /root/hyperscan/src/nfa/mcclellan.c:138

138 u8 cprime = m->remap[*c]; Missing separate debuginfos, use: debuginfo-install file-libs-5.11-31.el7.x86_64 glib2-2.42.2-5.el7.x86_64 glibc-2.17-196.tl2.3.x86_64 gmime-2.6.23-1.el7.x86_64 gpgme-1.3.2-5.el7.x86_64 libassuan-2.1.0-3.el7.x86_64 libcap-ng-0.7.5-4.el7.x86_64 libffi-3.0.13-16.el7.x86_64 libgcc-4.8.5-39.tl2.1.x86_64 libgpg-error-1.12-3.el7.x86_64 libpcap-1.5.3-8.el7.x86_64 libselinux-2.2.2-6.el7.x86_64 libstdc++-4.8.5-39.tl2.1.x86_64 libyaml-0.1.4-11.el7_0.x86_64 luajit-2.0.4-3.el7.x86_64 lz4-1.7.5-2.tl2.x86_64 nspr-4.10.8-2.el7_1.x86_64 nss-3.19.1-19.el7_2.x86_64 nss-softokn-3.16.2.3-13.el7_1.x86_64 nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64 nss-util-3.19.1-4.el7_1.x86_64 numactl-libs-2.0.9-6.el7_2.x86_64 openssl-libs-1.0.2k-19.tl2.1.x86_64 pcre-8.32-15.el7.x86_64 re2-20160401-2.el7.x86_64 sqlite-3.7.17-8.el7.x86_64 xz-libs-5.1.2-12alpha.el7.x86_64 yaml-cpp-0.5.1-2.el7.x86_64 zlib-1.2.7-15.el7.x86_64 (gdb) bt

0 doNormal16 (mode=CALLBACK_OUTPUT, do_accel=0 '\000', s=1,

end=0x7fa59872864e <Address 0x7fa59872864e out of bounds>, c_inout=<synthetic pointer>, m=0x126eb600)
at /root/hyperscan/src/nfa/mcclellan.c:138

1 mcclellanExec16_i (mode=CALLBACK_OUTPUT, c_final=0x0, single=0 '\000', ctxt=0x7fa4b03bbbc0,

cb=0x7fa4eae75490 <roseAnchoredCallback>, offAdj=0, len=60, 
buf=0x7fa598728612 <Address 0x7fa598728612 out of bounds>, qstate=0x0, state=<synthetic pointer>, 
m=0x126eb600) at /root/hyperscan/src/nfa/mcclellan.c:274

2 nfaExecMcClellan16_Bi (single=0 '\000', context=0x7fa4b03bbbc0,

cb=0x7fa4eae75490 <roseAnchoredCallback>, length=60, 
buffer=0x7fa598728612 <Address 0x7fa598728612 out of bounds>, offset=0, n=0x126eb5c0)
at /root/hyperscan/src/nfa/mcclellan.c:763

3 nfaExecMcClellan16_B (n=0x126eb5c0, offset=0,

buffer=0x7fa598728612 <Address 0x7fa598728612 out of bounds>, length=60, 
cb=0x7fa4eae75490 <roseAnchoredCallback>, context=0x7fa4b03bbbc0)
at /root/hyperscan/src/nfa/mcclellan.c:971

4 0x00007fa4eae625bd in runAnchoredTableBlock (t=, scratch=0x7fa4b03bbbc0,

atable=<optimized out>) at /root/hyperscan/src/rose/block.c:67

5 roseBlockAnchored (scratch=0x7fa4b03bbbc0, t=0x126d6580) at /root/hyperscan/src/rose/block.c:212

6 roseBlockExec (t=, scratch=) at /root/hyperscan/src/rose/block.c:395

7 0x00007fa4ead93f9e in rawBlockExec (scratch=0x7fa4b03bbbc0, rose=0x126d6580)

at /root/hyperscan/src/runtime.c:188

8 hs_scan (db=, data=, length=2644, flags=,

scratch=0x7fa4b03bbbc0, onEvent=<optimized out>, userCtx=0x7fa4c5e6d290)
at /root/hyperscan/src/runtime.c:419

9 0x00000000006bed9c in SCHSSearch (mpm_ctx=, mpm_thread_ctx=,

pmq=<optimized out>, buf=<optimized out>, buflen=<optimized out>) at util-mpm-hs.c:938

10 0x000000000058c04a in StreamMpmFunc (cb_data=, data=,

data_len=<optimized out>) at detect-engine-payload.c:64

11 0x000000000067af5c in StreamReassembleRawInline (progress_out=0x7fa4b03b8580, cb_data=0x7fa4c5e6db70,

Callback=0x58c010 <StreamMpmFunc>, p=0x7fa4b032cf60, ssn=<optimized out>)
at stream-tcp-reassemble.c:1487

12 StreamReassembleRaw (ssn=, p=p@entry=0x7fa4b028be90,

Callback=Callback@entry=0x58c010 <StreamMpmFunc>, cb_data=cb_data@entry=0x7fa4c5e6db70, 
progress_out=progress_out@entry=0x7fa4b03b8580, 
respect_inspect_depth=respect_inspect_depth@entry=false) at stream-tcp-reassemble.c:1677

13 0x000000000058c1e8 in PrefilterPktStream (det_ctx=0x7fa4b03b8530, p=0x7fa4b028be90, pectx=0x4749470)

at detect-engine-payload.c:83

14 0x000000000058f711 in Prefilter (det_ctx=det_ctx@entry=0x7fa4b03b8530, sgh=0xd415110,

p=p@entry=0x7fa4b028be90, flags=<optimized out>) at detect-engine-prefilter.c:169

15 0x0000000000557c33 in DetectRunPrefilterPkt (tv=0x9736360, scratch=0x7fa4c5e6dc70, p=0x7fa4b028be90,

det_ctx=0x7fa4b03b8530, de_ctx=0x470a9d0) at detect.c:734

16 DetectRun (th_v=th_v@entry=0x9736360, de_ctx=, det_ctx=0x7fa4b03b8530,

p=p@entry=0x7fa4b028be90) at detect.c:132

17 0x0000000000559757 in DetectRun (p=0x7fa4b028be90, det_ctx=, de_ctx=,

th_v=0x9736360) at detect.c:1810

18 DetectNoFlow (p=, det_ctx=, de_ctx=, tv=)

at detect.c:1810

19 Detect (tv=tv@entry=0x9736360, p=p@entry=0x7fa4b028be90, data=data@entry=0x7fa4b03b8530,

pq=pq@entry=0x0, postpq=postpq@entry=0x0) at detect.c:1870

20 0x00000000005eef5b in FlowWorker (tv=0x9736360, p=0x7fa4b028be90, data=0x7fa4b02ab430,

preq=0x5e9bfc0, unused=<optimized out>) at flow-worker.c:346

21 0x0000000000680e0b in TmThreadsSlotVarRun (tv=tv@entry=0x9736360, p=p@entry=0x7fa4b028be90,

slot=slot@entry=0x5e9d3a0) at tm-threads.c:143

22 0x0000000000661e2c in TmThreadsSlotProcessPkt (p=0x7fa4b028be90, s=0x5e9d3a0, tv=0x9736360)

at tm-threads.h:147

23 ReceiveCFWLoop () at source-cfw.c:378

24 0x0000000000681ee2 in TmThreadsSlotPktAcqLoop (td=0x9736360) at tm-threads.c:346

25 0x00007fa4e9636e25 in start_thread () from /lib64/libpthread.so.0

---Type to continue, or q to quit---

26 0x00007fa4e8f4935d in clone () from /lib64/libc.so.6

[xiangwang1]

It's difficult to find the root cause by only looking at the debug trace. It'll be good if you can help us to reproduce this case by having configurations including machine, OS, Suricata version & rules, etc.

[worldpeace365]

suricata.yaml.txt attached file is suricata config. suricata version is 4.1.0. mode is IPS worker. rules is th latest on suricata web site. receiver packets from rte_ring added by myself. Traffics is produced by T-rex， with http connects of 2000 cps and get 66636Bytes per connect. cpu info: [root@VM-0-49-centos ~/txfw]# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Thread(s) per core: 1 Core(s) per socket: 8 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50GHz Stepping: 5 CPU MHz: 2494.134 BogoMIPS: 4988.26 Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 4096K L3 cache: 36608K NUMA node0 CPU(s): 0-7 NUMA node1 CPU(s): 8-15 OS: centos 4.14.105 on VM of KVM

[worldpeace365]

It's difficult to find the root cause by only looking at the debug trace. It'll be good if you can help us to reproduce this case by having configurations including machine, OS, Suricata version & rules, etc.

Hi，I have commit detail info， Please help to process it. Thank you.