crash with stack guard in the main thread when the jni module is loaded

[davidnich]

stack guard calculates the end position of the stack when the thread is started, or in the case of the main thread, when the Qore library is initialized.

when the JVM sets up its own stack guard and enforces a lower limit, a crash occurs - ex:

2.572846: TID 1: check_stack() current: 0x7fffffff2bf0 limit: 0x7fffff816880 start: 0x7fffffffc880
Process 34762 stopped
* thread #1, name = 'qore/1', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
    frame #0: 0x00007fffcce1762d
->  0x7fffcce1762d: movl   (%rsi), %eax
    0x7fffcce1762f: leaq   0x30(%rbp), %rsi
    0x7fffcce17633: movl   $0x10000, %eax            ; imm = 0x10000 
    0x7fffcce17638: andl   0x4(%rsi), %eax
(lldb) cont
Process 34762 resuming
36.62901: TID 1: check_stack() current: 0x7fffffff1000 limit: 0x7fffff816880 start: 0x7fffffffc880
...
39.786629: TID 1: check_stack() current: 0x7ffffff03d30 limit: 0x7fffff816880 start: 0x7fffffffc880
Process 34762 stopped
* thread #1, name = 'qore/1', stop reason = signal SIGSEGV: address access protected (fault address: 0x7ffffff02de8)
    frame #0: 0x00007ffff6880594 libc.so.6`__printf_buffer + 36
libc.so.6`__printf_buffer:
->  0x7ffff6880594 <+36>: movq   %rsi, -0x4f8(%rbp)
    0x7ffff688059b <+43>: movl   $0x25, %esi
    0x7ffff68805a0 <+48>: movl   %ecx, -0x4dc(%rbp)
    0x7ffff68805a6 <+54>: movq   %fs:0x28, %rax

[davidnich]

This could be a glibc issue on Fedora 38 - currently it appears to not extend the stack size beyond 1MB even though ulimit -s is 8192 (8MB)

[davidnich]

This happens when the jni is loaded and the JVM sets its own stack guard implementation that sets a limit of 1MB on the stack