Crash in TLS socket finalizer

Leni-Vienne commented 3 months ago

How can we reproduce the crash?

Call some express routes a few times. They have postgres queries for most of them, using the 'pg' package. This crash does not happen in bun 1.20. I would like to test bun 1.21 but I am behind a company proxy so downgrading via the curl command doesn't work. Interestingly, it says I am on Bun Canary despite the fact I just uninstalled and re-installed Bun

Relevant log output

============================================================
Bun Canary v1.1.22-canary.96 (df33f2b2) Windows x64 (baseline)
CPU: sse42
Args: "D:\Users\leni.vienne\.bun\bin\bun.exe" "--env-file=../.env" "--hot" "run" "server.js"
Features: jsc dotenv http_server transpiler_cache(4) tsconfig(7) no_avx no_avx2
Builtins: "bun:main" "node:assert" "node:async_hooks" "node:buffer" "node:crypto" "node:dns" "node:events" "node:fs" "node:http" "node:net" "node:path" "node:querystring" "node:stream" "node:string_decoder" "node:tls" "node:tty" "node:url" "node:util" "node:util/types" "node:zlib"
Elapsed: 738032ms | User: 2406ms | Sys: 1390ms
RSS: 0.26GB | Peak: 0.26GB | Commit: 0.36GB | Faults: 149457

Stack Trace (bun.report)

Bun v1.1.22 (df33f2b) on windows x86_64_baseline [RunCommand]

Segmentation fault at address 0x2F00000057

socket.c:121: us_socket_is_closed
uws.zig:111: isClosed
Weak.h:82: JSC::Weak<WebCore::JSTLSSocket>::impl
HeapCell.h:57: JSC::HeapCell::zap
MarkedBlockInlines.h:476: JSC::MarkedBlock::Handle::finishSweepKnowingHeapCellType<JSC::JSDestructibleObjectDestroyFunc>::<lambda_1>::operator()
MarkedBlockInlines.h:508: JSC::MarkedBlock::Handle::finishSweepKnowingHeapCellType<JSC::JSDestructibleObjectDestroyFunc>
JSDestructibleObjectHeapCellType.cpp:56: JSC::JSDestructibleObjectHeapCellType::finishSweep
SweepingScope.h:43: JSC::SweepingScope::~SweepingScope
FreeList.h:88: JSC::FreeList::allocationWillFail
LocalAllocator.cpp:196: JSC::LocalAllocator::tryAllocateWithoutCollecting

Jarred-Sumner commented 3 months ago

This crash is unrelated to express. Can you include the source code for us to reproduce it?

Leni-Vienne commented 3 months ago

Okay I've been struggling to narrow it down It can occur :

when I save a .js file in --hot mode (maybe because the server was processing a call?)
in the middle of the execution of a function
on a windows machine
in an openstack environment
even if there is only one API query for Bun to process

It rarely crashes twice on the same line, which makes it very hard to narrow it down. It feels like there is an instability somewhere. I don't know if my code will be of much help but If you want it, how can I give it to you privately?

Here is the crash from the openstack container, running the same code :

Bun v1.1.22 (df33f2b) on linux x86_64_baseline [AutoCommand]

Segmentation fault at address 0x00000000

uws.zig:100: TLSSocketClass__finalize
ZigGeneratedClasses.cpp:39835: WebCore::JSTLSSocket::destroy
ld-temp.o:0: JSC::PreciseAllocation::sweep
ld-temp.o:0: JSC::Heap::handleNeedFinalize
ld-temp.o:0: JSC::Heap::finishChangingPhase
ld-temp.o:0: JSC::Heap::runEndPhase
ld-temp.o:0: JSC::Heap::runCurrentPhase
ld-temp.o:0: WTF::ScopedLambdaFunctor<void (...), JSC::Heap::collectInMutatorThread()::$_0>::implFunction

And another example from right now, this one failed specifically at a postgres query, I had console.log before and after it :

Bun v1.1.22 (df33f2b) on windows x86_64_baseline [RunCommand]

Segmentation fault at address 0x00000041

1 unknown/js code
openssl.c:322: us_internal_ssl_socket_close
uws.zig:91: close
Weak.h:82: JSC::Weak<WebCore::JSTLSSocket>::impl
HeapCell.h:57: JSC::HeapCell::zap
MarkedBlockInlines.h:496: JSC::MarkedBlock::Handle::finishSweepKnowingHeapCellType<JSC::JSDestructibleObjectDestroyFunc>::<lambda_1>::operator()
MarkedBlockInlines.h:508: JSC::MarkedBlock::Handle::finishSweepKnowingHeapCellType<JSC::JSDestructibleObjectDestroyFunc>
JSDestructibleObjectHeapCellType.cpp:56: JSC::JSDestructibleObjectHeapCellType::finishSweep
SweepingScope.h:43: JSC::SweepingScope::~SweepingScope
FreeList.h:88: JSC::FreeList::allocationWillFail

A crash right after saving a .js file in --hot mode :

Bun v1.1.22 (df33f2b) on windows x86_64_baseline [RunCommand]

Segmentation fault at address 0x2F00000057

socket.c:121: us_socket_is_closed
uws.zig:111: isClosed
Weak.h:82: JSC::Weak<WebCore::JSTLSSocket>::impl
PreciseAllocation.h:153: JSC::PreciseAllocation::isLowerTierPrecise
MarkedSpace.cpp:236: JSC::MarkedSpace::sweepPreciseAllocations
[memory:3435](<https://github.com/oven-sh/bun/blob/df33f2b2a264912c7a742d0c03587082415fbba6/C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.40.33807/include/memory#L3435>): std::unique_ptr<JSC::IsoSubspace,std::default_delete<JSC::IsoSubspace> >::operator bool
Heap.cpp:2160: JSC::Heap::handleNeedFinalize
Heap.cpp:2168: JSC::Heap::handleNeedFinalize
Heap.cpp:1738: JSC::Heap::changePhase
Heap.cpp:1378: JSC::Heap::runCurrentPhase

Jarred-Sumner commented 3 months ago

I think @cirospaciari has a fix in #13176

if you go to the Buildkite link in there and click on artifacts under build-bun x64 windows you could test it out

Jarred-Sumner commented 3 months ago

Fixed in https://github.com/oven-sh/bun/pull/13176

oven-sh / bun