search

status-im / nim-chronos

Chronos - An efficient library for asynchronous programming
https://status-im.github.io/nim-chronos/docs/chronos
Apache License 2.0
350 stars 51 forks source link

Why does this usage of asyncSpawn and ThreadSignalPtr leak memory? #542

Open PhilippMDoerner opened 4 weeks ago

PhilippMDoerner commented 4 weeks ago

Heyho, still trying to write my multithreading library and ensuring correctness with address sanitizer. I've been running into memory leak issues of around 20KB with the way I have set up my code (meaning asyncSpawn on a Future at the start of a loop, and process the async-work that came from that at a later part in the loop while waitFor-ing ThreadSignalPtr).

I managed to condense it down into this minimal example, which I do not understand why that leaks:

import std/[os]
import chronos
import chronos/threadsync

proc cleanupThread*() {.gcsafe, raises: [].}=
  {.cast(gcsafe).}:
    try:
      when defined(gcOrc):
        GC_fullCollect() # from orc.nim. Has no destructor.
    except Exception as e:
      echo "Exception during cleanup: ", e[]

proc processAsync() {.async.} =
  await sleepAsync(1)
  echo "Stuff" 

proc threadProc(threadSignal: ThreadSignalPtr) {.thread.} =
  asyncSpawn processAsync()

  waitFor threadSignal.wait()

  cleanupThread()

var thread: Thread[ThreadSignalPtr]
let signal = new(ThreadSignalPtr)[]

createThread(thread, threadProc, signal)
sleep(10)
discard signal.fireSync()

joinThread(thread)
discard signal.close()

There is nothing that should leak there. All futures that exist from processAsync should be resolved once during the sleep call. You can see that in the console as well since Stuff will get printed there. So what is leaking?

Command to compile: nim r -f --debugger:native --cc:clang -d:useMalloc --passc:"-fsanitize=address -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" --passl:"-fsanitize=address -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" -d:release src/playground.nim

Address sanitizer stacktraces: ```txt 50823==ERROR: LeakSanitizer: detected memory leaks Direct leak of 256 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c796063 in nimNewObj /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/arc.nim:81:343 #2 0x599e1c7be8eb in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:752:88 #3 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #4 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #5 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #6 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #7 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #8 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #9 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #10 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #11 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #12 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #13 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #14 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 11272 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c799a5c in setLen::setLen(var>>>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:133:151 #3 0x599e1c79f581 in initTable::initTable(int) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/pure/collections/tableimpl.nim:170:208 #4 0x599e1c7b254f in new::new(typeDesc>, typeDesc) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/ioselects/ioselectors_epoll.nim:105:16 #5 0x599e1c7be751 in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:747:128 #6 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #7 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #8 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #9 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #10 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #11 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #12 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #13 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #14 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #15 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #16 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #17 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 2568 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c79b295 in setLen::setLen(var>>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:133:151 #3 0x599e1c7a0751 in initTable::initTable__OOZOOZOOZOnimbleZpkgs50Zchronos4552O48O504553ed50d495255ae52555448a52ec54f4849daf555752df555449b53505057afZchronosZinternalZasyncengine_u1757(int) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/pure/collections/tableimpl.nim:170:211 #4 0x599e1c7beac4 in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:759:24 #5 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #6 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #7 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #8 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #9 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #10 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #11 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #12 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #13 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #14 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #15 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #16 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 2056 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c79aa8a in setLen::setLen(var>>>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:133:151 #3 0x599e1c7a0451 in initTable::initTable__OOZOOZOOZOnimbleZpkgs50Zchronos4552O48O504553ed50d495255ae52555448a52ec54f4849daf555752df555449b53505057afZchronosZinternalZasyncengine_u1493(int) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/pure/collections/tableimpl.nim:170:211 #4 0x599e1c7bea5a in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:758:24 #5 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #6 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #7 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #8 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #9 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #10 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #11 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #12 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #13 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #14 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #15 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #16 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 1544 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c7991d5 in setLen::setLen(var>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:133:151 #3 0x599e1c7be99b in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:755:2 #4 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #5 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #6 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #7 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #8 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #9 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #10 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #11 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #12 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #13 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #14 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #15 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 1032 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c79a585 in setLen::setLen(var>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:133:151 #3 0x599e1c79a585 in newSeq::newSeq(var>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:140:2 #4 0x599e1c79a585 in newSeq::newSeq__OOZOOZOOZOnimbleZpkgs50Zchronos4552O48O504553ed50d495255ae52555448a52ec54f4849daf555752df555449b53505057afZchronosZinternalZasyncengine_u1254(range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system.nim:631:2 #5 0x599e1c7be9f6 in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:757:16 #6 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #7 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #8 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #9 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #10 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #11 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #12 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #13 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #14 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #15 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #16 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #17 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 344 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c796063 in nimNewObj /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/arc.nim:81:343 #2 0x599e1c7b2504 in new::new(typeDesc>, typeDesc) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/ioselects/ioselectors_epoll.nim:107:75 #3 0x599e1c7be751 in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:747:128 #4 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #5 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #6 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #7 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #8 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #9 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #10 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #11 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #12 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #13 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #14 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #15 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 104 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c7991d5 in setLen::setLen(var>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:133:151 #3 0x599e1c7be9c7 in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:756:2 #4 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #5 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #6 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #7 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #8 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #9 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #10 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #11 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #12 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #13 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #14 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #15 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 72 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c79a38b in setLen::setLen(var>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:133:151 #3 0x599e1c79a38b in newSeq::newSeq(var>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:140:2 #4 0x599e1c7b2641 in new::new(typeDesc>, typeDesc) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/ioselects/ioselectors_epoll.nim:110:2 #5 0x599e1c7be751 in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:747:128 #6 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #7 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #8 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #9 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #10 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #11 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #12 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #13 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #14 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #15 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #16 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #17 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 24 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c799f9b in setLen::setLen(var>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:133:151 #3 0x599e1c799f9b in newSeq::newSeq(var>, range09223372036854775807) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:140:2 #4 0x599e1c7b260f in new::new(typeDesc>, typeDesc) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/ioselects/ioselectors_epoll.nim:109:2 #5 0x599e1c7be751 in asyncengine::newDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:747:128 #6 0x599e1c7beebb in asyncengine::getThreadDispatcher /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1093:39 #7 0x599e1c7bfe4c in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1107:9 #8 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #9 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #10 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #11 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #12 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #13 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #14 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #15 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #16 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #17 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o Indirect leak of 16 byte(s) in 1 object(s) allocated from: #0 0x599e1c7426f1 in calloc (/home/isofruit/.cache/nim/playground_r/playground_CAED38D1B3F5F9549C0D683A5A5C9A6761A36907+0x1266f1) (BuildId: 45b58de06b1e762d7de2cf93447bf65a9b6087d7) #1 0x599e1c7939eb in newSeqPayload /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:44:773 #2 0x599e1c79bdff in add::add(var>>, sink>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/seqs_v2.nim:116:26 #3 0x599e1c7a4e9e in push::push(var>>, sink>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/pure/collections/heapqueue.nim:137:2 #4 0x599e1c7bff59 in asyncengine::setTimer(timer::Moment, proc, pointer) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncengine.nim:1110:2 #5 0x599e1c7cb54f in asyncfutures::sleepAsync(timer::Duration) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1397:163 #6 0x599e1c7d1a41 in asyncfutures::sleepAsync(int) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:1403:107 #7 0x599e1c7d1a41 in processAsync::processAsync(ref) /home/isofruit/dev/playground/src/playground.nim:14:27 #8 0x599e1c7cbd47 in asyncfutures::futureContinue(ref) /home/isofruit/.nimble/pkgs2/chronos-4.0.2-5ed2d147ae4760a4ec6f01daf794df761b5229af/chronos/internal/asyncfutures.nim:382:95 #9 0x599e1c7d2cde in playground::processAsync /home/isofruit/dev/playground/src/playground.nim:14:2 #10 0x599e1c7d3465 in playground::threadProc(ptr) /home/isofruit/dev/playground/src/playground.nim:18:15 #11 0x599e1c798579 in threadProcWrapDispatch::threadProcWrapDispatch(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:69:2 #12 0x599e1c798579 in threadProcWrapStackFrame::threadProcWrapStackFrame(ptr>>) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:95:2 #13 0x599e1c78ede8 in threadProcWrapper::threadProcWrapper(pointer) /home/isofruit/.choosenim/toolchains/nim-2.0.4/lib/system/threadimpl.nim:101:2 #14 0x599e1c679a36 in asan_thread_start(void*) asan_interceptors.cpp.o SUMMARY: AddressSanitizer: 19288 byte(s) leaked in 11 allocation(s). ```
PhilippMDoerner commented 4 weeks ago

Turns out you need to indeed call =destroy(getThreadDispatcher()) on the dispatcher and do so before you collect all cycles with GC_fullCollect()`. That solved it.

arnetheduck commented 4 weeks ago

Turns out you need to indeed call =destroy(getThreadDispatcher()) on the dispatcher and do so before you collect all cycles with GC_fullCollect()`. That solved it.

feel free to turn this into a documentation page outlining multithreaded / orc + chronos use

PhilippMDoerner commented 4 weeks ago

Actually, pretty solid advice. I'm repurposing this issue for a PR towards that change

PhilippMDoerner commented 3 weeks ago

For the purposes of decent examples with async-event-loop in other threads and cleaning up after them: Are there easy ways to tell chronos "process your remaining async work" ? I didn't get around to asking that one yet I think