jamiejennings
commented
7 years ago Steve, the change you want is to a constant that limits the amount of backtracking that a single pattern is able to do.
I will create a v0.99k-mod5 containing a change of this value from 400 to 800 (later today when I get back to the office). Meanwhile, if you want to do it yourself, instructions are below.
I do want to understand, at some point in time, how your patterns combine to need more than 400 possible backtracking points. It's certainly possibly with a lot of patterns that are combined into a grand "this or that or the next or the next..." pattern, and I'm interested in the use case.
If you want to make the change yourself, you can edit as shown below and then follow the 2 steps below. Here, I've changed it from 400 to 800 in rosie-pattern-language/submodules/rosie-lpeg/src/lptypes.h:
$ git --no-pager diff lptypes.h
diff --git a/src/lptypes.h b/src/lptypes.h
index 5eb7987..b031cbe 100644
--- a/src/lptypes.h
+++ b/src/lptypes.h
@@ -51,7 +51,7 @@
/* default maximum size for call/backtrack stack */
#if !defined(MAXBACK)
-#define MAXBACK 400
+#define MAXBACK 800
#endif
$ After editing as shown above, and while in the directory containing lptypes.h, run make and then copy the resulting lpeg.so to rosie's lib directory. (Substitute linux for macosx in the make command if needed.)
Note that running rosie's make clean and then make will reset all the submodules to the official versions, so you don't want to use those here.
$ make macosx
make lpeg.so "DLLFLAGS = -bundle -undefined dynamic_lookup"
gcc -Wall -Wextra -pedantic -Waggregate-return -Wcast-align -Wcast-qual -Wdisabled-optimization -Wpointer-arith -Wshadow -Wsign-compare -Wundef -Wwrite-strings -Wbad-function-cast -Wdeclaration-after-statement -Wmissing-prototypes -Wnested-externs -Wstrict-prototypes -Wunreachable-code -O2 -std=c99 -I../lua//include -fPIC -c -o lpvm.o lpvm.c
gcc -Wall -Wextra -pedantic -Waggregate-return -Wcast-align -Wcast-qual -Wdisabled-optimization -Wpointer-arith -Wshadow -Wsign-compare -Wundef -Wwrite-strings -Wbad-function-cast -Wdeclaration-after-statement -Wmissing-prototypes -Wnested-externs -Wstrict-prototypes -Wunreachable-code -O2 -std=c99 -I../lua//include -fPIC -c -o lpcap.o lpcap.c
gcc -Wall -Wextra -pedantic -Waggregate-return -Wcast-align -Wcast-qual -Wdisabled-optimization -Wpointer-arith -Wshadow -Wsign-compare -Wundef -Wwrite-strings -Wbad-function-cast -Wdeclaration-after-statement -Wmissing-prototypes -Wnested-externs -Wstrict-prototypes -Wunreachable-code -O2 -std=c99 -I../lua//include -fPIC -c -o lptree.o lptree.c
gcc -Wall -Wextra -pedantic -Waggregate-return -Wcast-align -Wcast-qual -Wdisabled-optimization -Wpointer-arith -Wshadow -Wsign-compare -Wundef -Wwrite-strings -Wbad-function-cast -Wdeclaration-after-statement -Wmissing-prototypes -Wnested-externs -Wstrict-prototypes -Wunreachable-code -O2 -std=c99 -I../lua//include -fPIC -c -o lpcode.o lpcode.c
env gcc -bundle -undefined dynamic_lookup rcap.o rbuf.o lpvm.o lpcap.o lptree.o lpcode.o -o lpeg.so
$ cp lpeg.so ../../../lib/
yausteve-ibm
Hi, Jamie, I encountered the following error when loading MANIFEST: Traceback (most recent call last): File "/opt/log-analytics/bin/mytest", line 33, in
extract()
File "/opt/log-analytics/bin/mytest", line 31, in extract
r = engine.load_manifest("$sys/vmware.MANIFEST")
File "/opt/log-analytics/module/rosie.py", line 154, in load_manifest
retvals = self.rosie.get_retvals(r)
File "/opt/log-analytics/module/rosie.py", line 102, in get_retvals
return self._get_retvals(messages, self.rosie.rosieL_free_stringArray)
File "/opt/log-analytics/module/rosie.py", line 113, in _get_retvals
raise RuntimeError(retvals[1]) # exception indicating that the call failed
RuntimeError: src/core/engine.lua:121: backtrack stack overflow (current limit is 400)
It seems this happens when I have a large number of patterns defined in a number of rpl files. Could you advise whether there is a way to workaround this? Appreciate your advice. Many thanks. -Steve