mratsim / weave

A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead
Other
541 stars 21 forks source link

GC Boehm Error with more than 8 threads #167

Open Shadowfall357 opened 3 years ago

Shadowfall357 commented 3 years ago

Hello,

is it possible to use Weave with gc Boehm? I wrote this simple program:

import weave
import os

proc main(): void =
    putEnv("WEAVE_NUM_THREADS", "9")
    init(Weave)
    exit(Weave)

main()

Unfortunately I get this error when compiling with Boehm GC with more than 8 threads:

Error: unhandled exception: /home/.nimble/pkgs/weave-0.4.0/weave/instrumentation/contracts.nim(86, 15)

localCtx == default(TLContext)
    Contract violated for pre-condition at scheduler.nim:252
        localCtx == default(TLContext)
    The following values are contrary to expectations:
        (worker: (ID: 8, left: -1, right: -1, parent: 3, workSharingRequests: (front: 0, back: 0, buffer: ...), deque: (pendingTasks: 0, head: ..., tail: (fn: ..., parent: ..., prev: ..., next: ..., start: 0, cur: 0, stop: 0, stride: 0, futures: ..., futureSize: 0, hasFuture: false, isLoop: false, isInitialIter: false, data: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])), currentTask: ..., leftIsWaiting: true, rightIsWaiting: true, isWaiting: false), thefts: (outstanding: 0, dropped: 0, rng: (s0: 11568752105299578744, s1: 5231855230099380833, s2: 13070411628727900855, s3: 17390588265438726597), stealHalf: false, recentTasks: 0, recentThefts: 0), taskCache: (top: ..., freeFn: ..., count: 0, recentAsk: 0, registeredAt: ...), stealCache: (stack: ..., rawMem: ..., len: 1), runtimeIsQuiescent: false, signaledTerminate: false) == default(TLContext)  [Worker 8]
 [AssertionDefect]

Am I doing something wrong or does the boehm gc not work with weave?

Thank you in advance.

mratsim commented 3 years ago

I can reproduce. It's both an easy and hard fix.

So my check is a sanity check to ensure that thread-local variables are all zeroes. I technically only rely on this only for my background Weave experimental API https://github.com/mratsim/weave/blob/f41a5626880e55600f57eb1dbf5c29ae30e8cfb2/weave/datatypes/context_thread_local.nim#L107-L110, https://github.com/mratsim/weave/blob/e5a3701d59ec17c1c71a22a299ab526777054b44/weave/contexts.nim#L34

In practice, this is true for all other GCs, and it seems to be true whether TLS emulation is used or not but:

  1. GcBoehm has strange comparison issues with thread-local variables\ before the offending check, it seems like I do get binary zero image image \ Another strange error if I remove that check and run the test suite on Boehm image

  2. It seems like GC Boehm somehow implies TLS Emulation which I don't support because it does weird things and I can't deactivate it.

So the next steps are:

  1. Fixing Nim so that GC Boehm doesn't imply TLS emulation
  2. Check that the issue is fixed.

Note that I should be able to rely on zero initialization of thread-local variables in C++: https://en.cppreference.com/w/cpp/language/zero_initialization image I assume it's the same for C though I can't find a public spec.

Araq commented 3 years ago

--gc:boehm does indeed imply --tlsEmulation:on, this was a bugfix as Boehm apparently doesn't consider roots inside thread-local storage. However, this is likely to be platform specific...