Open vtjnash opened 8 months ago
c42f
commented
8 months ago I can't reproduce this on my dev machine with the example definition of is_operator_start_char(). Could the nanosolider machine have a small max stack size?
I thought that in practice we'd be safe until expressions got much more deeply nested. The flisp parser uses exactly the same recursive structure but it's run in an interpreter so I guess it might not be using the native stack...
c42f
commented
8 months ago A nesting depth of 1700 is sufficient to cause stack overflow on my machine. For example:
julia> make_expr(N) = N == 0 ? UInt32(0) : :(u == $(UInt32(N)) || $(make_expr(N-1)))
julia> parsestmt(Expr, string(make_expr(1700)));
ERROR: StackOverflowError:
...Typing such huge expressions is unusual and probably cursed. But if someone is generating them it's disturbingly easy to hit the limit here.
KristofferC
commented
8 months ago I think I saw this error message locally when doing a precompile and it also involved this code is_operator_start_char(u::UInt32) = u == 0x00000021 || (u == 0x00000024.... So it might be that that function is on the border of overflowing?
oscardssmith
commented
8 months ago I've been seeing this a lot too.
Precompiling project...
221 dependencies successfully precompiled in 330 seconds. 41 already precompiled.
1 dependency had output during precompilation:
┌ Tokenize
│ ┌ Error: JuliaSyntax parser failed — falling back to flisp!
│ │ exception = (StackOverflowError(), Union{Ptr{Nothing}, Base.InterpreterIP}[Ptr{Nothing} @0x00007f178ebd056a, Ptr{Nothing} @0x00007f178e91c08c, Ptr{Nothing} @0x00007f178e91c643, Ptr{Nothing} @0x00007f178eca7a19, Ptr{Nothing} @0x00007f178ead8d2f, Ptr{Nothing} @0x00007f178ee75f33, Ptr{Nothing} @0x00007f178ec82f81, Ptr{Nothing} @0x00007f178ec8303e, Ptr{Nothing}
...
vtjnash
commented
8 months ago (Aside: why does these errors keep printing out a weird backtrace()-looking thing instead of the usual catch_stacktrace()?)
KristofferC
commented
8 months ago KristofferC
commented
8 months ago This code is from Tokenize I guess so we could change that code as an immidiate workaround I guess. Or is it even from this package?
c42f
commented
8 months ago It's from Tokenize. In JuliaSyntax I've changed the way that function is defined for efficiency and maintainability.
KristofferC
commented
8 months ago Ok, I'll port that and update it.
KristofferC
commented
8 months ago
observed in https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_hash/5a9df24_vs_3250804/ODE.primary.log
In general, a compiler/parser commonly must not use recursion since it is surprisingly easy for the user to generate inputs that cause it to crash.