scan_prefix has no effective recursion limit

[NWilson]

I run this:

./pcre2test -q -8 -jit
  re> /X?(R||){3335}/

I get a crash due to stack overflow; excessive consumption of system stack, due to recursive calls to scan_prefix. The problem is more easily reproducible if you use a configuration which either reduces the stack size, or uses more than normal.

For example, I found this problem while running the unit tests with Clang ASAN, which bloats the binary with extra stuff, so recursion chews up more stack space.

However, a completely default build of PCRE2 can cause the segfault if you do e.g. ulimit -s 512 (512KiB) to reduce the stack down to something smaller than the Linux default.

[zherczeg]

It has a call limit but not a depth limit. Do you want to add it or shall I do it?

[NWilson]

I'd be grateful if you could do it, although I would be able to. I haven't yet even worked out what scan_prefix is doing.

Does it need to consume more than ~10 characters? If so, then it should probably be converted from recursion into some sort of loop instead. Simply adding an extra argument to the function, to track call depth, would limit the string length that it could consume.

[zherczeg]

It consumes max 12 characters at the moment. Those characters can be preceded by lot of control opcodes, such as ? | + (). I am able to use a single while loop at the end, but that is far from simple.