Closed questor closed 1 month ago
update: after updating my codebase the crash is gone, I think it's detecting the number of threads correct now. but still, if you start tundra with "-j 65" it crashes, so I will keep this ticket open, even when it's not happening for me by default anymore.
hm, no idea what is going on, here https://github.com/deplinenoise/tundra/blob/e48e03bbd8a193889337d25b42cfbd64c64c1c33/src/BuildQueue.cpp#L742 the number of threads is clamped, but still (as seen in the logs) more threads are created than the clamped number (and it will crash because there is not enough space for all the threads and memory will be overwritten). I've compiled tundra with clang, is it an error in the alias analysis of the compiler and with gcc it's working?
No idea, but I'll try to repro this
At first glance the code looks correct to me
Yeah I can sort of repro on this random Ubuntu machine I have, compiling with gcc. I don't create more threads than 64 even if I specify more on the command line, but it does crash with a stack smash protection when it happens:
[D] joining with build thread 57
[D] joining with build thread 58
[D] joining with build thread 59
[D] joining with build thread 60
[D] joining with build thread 61
[D] joining with build thread 62
[D] joining with build thread 63
*** stack smashing detected ***: terminated
Aborted (core dumped)
This doesn't happen with -j 64 so something is up.
after upgrading my linux mint to the newest version with ubuntu 24.04 as base for unknown reasons tundra thinks I have a machine with 128 cores and crashes. I see that the max threads is limited to 64 and right afterwards it crashes. when taking debugging-logs I can see that 128 threads are reported (dunno why, I'm using a virtual machine with 2 or 4 cores configured), 128 threads are created and then it crashes.
manually setting number of threads to a low number fixes the issue, but using more than 64threads will crash tundra after it clamped the number of threads to 64... so it's easy reproducable by setting the threadnumber to 65 and let tundra run.
the complete log is here:
and the call-stack: t2::BuildQueueBuildNodeRange(t2::BuildQueue, int, int, int) t2::DriverBuild(t2::Driver) ?? ??