microsoft / ClearScript

A library for adding scripting to .NET applications. Supports V8 (Windows, Linux, macOS) and JScript/VBScript (Windows).
https://microsoft.github.io/ClearScript/
MIT License
1.74k stars 148 forks source link

Crash "ClearScriptV8.win-x64.dll" randomly when stressing with many threads v7.3.1.0 #411

Closed GameHackerPM closed 2 years ago

GameHackerPM commented 2 years ago

Hello, I'm currently facing a crashing issue using ClearScript while starting 50 threads at the same time together! All I got is just my Program stopped working and need to be closed! Then I found in Event Viewer the following details of the error:

Faulting application name: TMPicker.exe, version: 2.0.0.0, time stamp: 0x6321f840 Faulting module name: ClearScriptV8.win-x64.dll, version: 7.3.1.0, time stamp: 0x62bb461e Exception code: 0xc0000005 Fault offset: 0x00000000002efbbc Faulting process id: 0x13f0 Faulting application start time: 0x01d8c874fb7943a0 Faulting application path: C:\xxxx\TMPicker.exe Faulting module path: C:\xxxx\ClearScriptV8.win-x64.dll Report Id: 6c5f9aaa-757a-4f76-b65e-dd1c468d3196 Faulting package full name: Faulting package-relative application ID:

Please let me know if you need any other information or files for it to find the issue! Thanks.

ClearScriptLib commented 2 years ago

Hi @GameHackerPM,

In the absence of additional information, the most likely cause of a 64-bit V8 crash is heap exhaustion.

By default, each V8 runtime uses a fixed-size heap, and if that heap is so full that the garbage collector can't free up sufficient space for whatever the script is trying to do, V8 forcefully crashes the process.

Of course, the crash could be due to something entirely different in your case. If you don't believe that heap exhaustion is a possibility, see if you can capture and share a crash dump. We'll be happy to take a look!

Collecting User-Mode Dumps

Thanks!

GameHackerPM commented 2 years ago

Hi @ClearScriptLib,

Thanks for your reply.. I'm not sure if it could be from heap exhaustion or something else, but it's not after I run the program for long time, it's just when I try to run multiple threads at the same time, it wasn't happening before with version 7.2.5, but currently I can see it happens with the current version! It's about something gets wrong when creating multiple instance and executes codes at the same time I believe so. The problem is this happening totally randomly! Like I can't reproduce it easily tbh, but I will try to get a crash dump if I got a chance. I also made a dump with TaskManager but it's a big file (700+MB) if that can be helpful, or I will try to get that crash dump created and send it to you.

Also to make sure, is these dump files are sensitive or showing any of the info used in the application, like I don't wanna share these info with the public, just let me know if that may be bad to put it here and there is another way privately to share when I get it ready.

Thanks again.

GameHackerPM commented 2 years ago

Hi @ClearScriptLib, Here is the required file: https://easyupload.io/xnqee8 Waiting for your reply..

ClearScriptLib commented 2 years ago

Sorry, that link isn't working for us:

image

GameHackerPM commented 2 years ago

Sorry about that, I think their server have something wrong with it. Here is from another site: https://file.io/tACeZ4eWzuNM

ClearScriptLib commented 2 years ago

Hi @GameHackerPM,

Thanks for sharing your crash dump!

It looks like you're calling V8ScriptEngine.CollectGarbage, and V8 is crashing deep inside its garbage collector:

         v8::internal::SlotSet::Iterate<`lambda at ../../../src/heap/mark-compact.cc:4526:11',`lambda at ../../..\src/heap/slot-set.h:348:20'>(unsigned __int64 chunk_start, unsigned __int64 start_bucket, unsigned __int64 end_bucket, v8::internal::RememberedSetUpdatingItem<v8::internal::MajorNonAtomicMarkingState,1>::UpdateUntypedPointers::<lambda_1> callback, v8::internal::SlotSet::Iterate<`lambda at ../../../src/heap/mark-compact.cc:4526:11'>::<lambda_1> empty_bucket_callback) Line 489    C++
[Inline] v8::internal::SlotSet::Iterate(unsigned __int64 chunk_start, unsigned __int64 start_bucket, unsigned __int64 end_bucket, v8::internal::RememberedSetUpdatingItem<v8::internal::MajorNonAtomicMarkingState,1>::UpdateUntypedPointers::<lambda_1> mode, v8::internal::SlotSet::EmptyBucketMode) Line 347 C++
[Inline] v8::internal::RememberedSetOperations::Iterate(v8::internal::SlotSet * slot_set, v8::internal::MemoryChunk * chunk, v8::internal::RememberedSetUpdatingItem<v8::internal::MajorNonAtomicMarkingState,1>::UpdateUntypedPointers::<lambda_1> mode, v8::internal::SlotSet::EmptyBucketMode) Line 42   C++
[Inline] v8::internal::RememberedSet<0>::Iterate(v8::internal::MemoryChunk * chunk, v8::internal::RememberedSetUpdatingItem<v8::internal::MajorNonAtomicMarkingState,1>::UpdateUntypedPointers::<lambda_1> callback, v8::internal::SlotSet::EmptyBucketMode mode) Line 174  C++
         v8::internal::RememberedSetUpdatingItem<v8::internal::MajorNonAtomicMarkingState,1>::UpdateUntypedPointers() Line 4534    C++
         v8::internal::RememberedSetUpdatingItem<v8::internal::MajorNonAtomicMarkingState,1>::Process() Line 4466  C++
[Inline] v8::internal::PointersUpdatingJob::UpdatePointers(v8::JobDelegate * delegate) Line 4365    C++
         v8::internal::PointersUpdatingJob::Run(v8::JobDelegate * delegate) Line 4351  C++
         v8::platform::DefaultJobState::Join() Line 110    C++
         v8::platform::DefaultJobHandle::Join() Line 225   C++
         v8::internal::MarkCompactCollector::UpdatePointersAfterEvacuation() Line 4804 C++
         v8::internal::MarkCompactCollector::Evacuate() Line 4271  C++
         v8::internal::MarkCompactCollector::CollectGarbage() Line 631 C++
         v8::internal::Heap::MarkCompact() Line 2605   C++
         v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector collector, v8::internal::GarbageCollectionReason gc_reason, const char * collector_reason, const v8::GCCallbackFlags gc_callback_flags) Line 2303 C++
         v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace space, v8::internal::GarbageCollectionReason gc_reason, const v8::GCCallbackFlags gc_callback_flags) Line 1859   C++
         v8::internal::Heap::CollectAllAvailableGarbage(v8::internal::GarbageCollectionReason gc_reason) Line 1642 C++
         v8::Isolate::LowMemoryNotification() Line 9266    C++
[Inline] V8IsolateImpl::LowMemoryNotification() Line 412    C++
[Inline] V8IsolateImpl::CollectGarbage(bool) Line 854   C++
         V8ContextImpl::CollectGarbage(bool exhaustive) Line 953   C++
         V8Context_CollectGarbage(const V8EntityHandle<V8Context> & handle, char exhaustive) Line 1495 C++

Since ClearScript plays no role in V8 heap management, and assuming that no other native code is corrupting memory, it's likely that you're hitting a V8 bug.

If this were something we could reproduce, we'd report the issue to the V8 team, so if you have a minimal code sample that reproduces the crash without revealing sensitive information, please share it.

In the meantime, we have a couple of things you could try.

First, consider getting rid of the CollectGarbage calls. The V8 team recommends against explicit GC invocation.

Second, try disabling V8's background operations. Doing so makes V8 behave more predictably at a very slight performance cost. Here's how to set that up – and make sure this code runs before you create any V8 script engines or runtimes:

V8Settings.GlobalFlags |= V8GlobalFlags.DisableBackgroundWork;

Good luck, and please send any new thoughts or observations our way!

GameHackerPM commented 2 years ago

Hi @ClearScriptLib,

Thanks for your reply.

I have taken with your advice and applied the changes you said, will watch it for today to see if it gonna crash again or not. Also will try to create a small console application which do the same thing my program do for the ClearScript part, the problem is it's totally randomly happening! Like will need it to crash at least once to send it to you so you don't get confused that everything is working. I know how is it when we can't reproduce the issue, totally understand that. So will try to figure something out for you.

Tonight, I'm gonna reply with the updates on the new changes I applied that you told me to do, and will let you know if it crashed again or not.

Thanks again.

GameHackerPM commented 2 years ago

It seems like the advice/changes you told me working fine as it didn't crash with me after applying these changes so far, but couldn't create a small program to reproduce it, so I don't know the best way to reproduce it so you can fix anyway!

Thanks.

ClearScriptLib commented 2 years ago

Hi @GameHackerPM,

It's great to hear that you found a solution!

BTW, much of V8's background work has to do with heap management, so the crash is most likely due to a race condition between that and the forced GC. Disabling V8's background work probably makes forced GC safe again, so if it provides a measurable benefit, then by all means go for it.

Please feel free to reopen this issue if you have additional thoughts or findings on this topic, or if you'd like to share a crashing code sample.

Thank you!