Seq.exe crash with stack overflow applying signal with many filters (2023.4+)

[MaceWindu]

Describe the bug We have a signal with multiple filters (don't know exact number, but should be 50-100 filters) organized as:

• one filter to include events
• many filters to exclude events from signal by some condition

At some point after adding additional filter seq.exe started to crash with following error:

Faulting application name: seq.exe, version: 2024.1.11001.0, time stamp: 0x65680000
Faulting module name: flarec.dll, version: 0.0.0.0, time stamp: 0x65c17e29
Exception code: 0xc00000fd
Fault offset: 0x00000000000b8aa7
Faulting process id: 0x3348
Faulting application start time: 0x01da5b437cea1b06
Faulting application path: C:\Program Files\Seq\seq.exe
Faulting module path: C:\Program Files\Seq\Native\flarec.dll
Report Id: 2ede6ee7-6cc9-446f-8657-8888633c83cc
Faulting package full name: 
Faulting package-relative application ID: 

Crashes started with 2023.4 version to the point where we start service and it immediately crash, so first thing we do is update to 2024.1 which presumably helped. But not for long as after couple of more filters it started to crash again.

As workaround currently we removed several filters we don't need anymore to keep it working.

BTW, it would be nice to have better editor for signal filters as current one is not suitable for signal with many/big filters.

Environment (please complete the following information):

• OS: Windows
• Seq Version: 2024.1.11001.0
• Using Docker? No

[nblumhardt]

Thanks for the report, we'll look into this 👍

[nblumhardt]

Hi @MaceWindu, would it be possible to please share the JSON signal definition in question (via Signal - Edit as JSON) via our support address support@datalust.co, with any sensitive information redacted? This will help ensure we're looking in the right place. Thanks!

[jorgenfries]

I created a different issue with the same problem https://github.com/datalust/seq-tickets/discussions/2092 Feel free to merge or close my ticket

[nblumhardt]

Thanks @jorgenfries. Is there a chance you could mail support@datalust.co with more information about what's on the dashboard causing the issue? We're keen to narrow this down as much as possible so any/all detail you can provide would be welcome. Thanks again, and sorry about the disruption!

[jorgenfries]

@nblumhardt Sent an email your way

[MaceWindu]

@nblumhardt sent you email

[MaceWindu]

@jorgenfries your issue could be a different one as 0xc0000005 is ACCESS VIOLATION (but it also could be caused by stack overflow in theory)

[nblumhardt]

Just a quick update on this, we have a patch, but we're working to ensure that it doesn't regress any other scenarios. ETA should be within a week or so - thanks for your patience, and sorry about the disruption.

[nblumhardt]

We're introducing some new settings in 2024.2 that can be used to protect against the possibility of this overflow, at the expense of disallowing some previously acceptable queries and filters.

On Windows those can be set to conservative values with:

seq config set -k runtime.expressionNestingLimit -v 90
seq config set -k runtime.nativeExpressionNestingLimit -v 250
seq service restart

It's likely, once we've collected some additional metrics and telemetry, that new Seq installs will carry lower default values for these settings in future. Until then, this will unfortunately persist as a known limitation.

For practical purposes, on Windows where the issue is most likely to occur (lower default stack size), creating a DWORD DefaultStackSize key under HKLM\Software\Microsoft\.NETFramework, with the value 200000 (hex) will add enough additional stack space to work around this in most situations.