search

Chia-Network / chia-blockchain

Chia blockchain python implementation (full node, farmer, harvester, timelord, and wallet)
Apache License 2.0
10.83k stars 2.02k forks source link

[Bug] Slow Bluebox Timelord - Child Process Terminated #18513

Open thesemaphoreslim opened 3 weeks ago

thesemaphoreslim commented 3 weeks ago

What happened?

When attempting to run a slow bluebox timelord in Windows 11, the debug.log outputs repeat "timelord chia.timelord.timelord : ERROR Exception manage discriminant queue: A child process terminated abruptly, the process pool is not usable anymore" errors when receiving uncompacted proofs from the full node. Attached to this ticket you will find the debug.log (DEBUG logging enabled) and my config.yaml. This occurs when starting the services with "chia start timelord-only" as well as "chia start timelord-launcher-only timelord-only".
debug.zip

I've confirmed the bug does NOT exist on Linux distributions (specifically Ubuntu server 22.04).

Version

2.4.3

What platform are you using?

Windows

What ui mode are you using?

CLI

Relevant log output

Logs attached.
Motophan commented 3 weeks ago

I have this issue also

thesemaphoreslim commented 2 weeks ago

Update: This issue appears to only occur when using the installer for Windows available from chia.net/downloads. When installing from source on Windows, the slow bluebox timelord functions as expected. Perhaps a dependency or other component is missing from the installer.

wjblanke commented 2 weeks ago

What CPU are you running? Can we get the crash log. Thinking it might be an illegal instruction.

thesemaphoreslim commented 2 weeks ago

Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz 2.59 GHz

thesemaphoreslim commented 2 weeks ago

What CPU are you running? Can we get the crash log. Thinking it might be an illegal instruction.

DumpStack.log

I've attached the crash log. I understand you guys are finding it difficult to duplicate the problem so here is some additional information.

I'm launching the "slow" bluebox timelord on Windows devices with the following command: chia start timelord-only

I've already provided the config.yaml and it is the same one I use for installations from source (which result in a fully working "slow" bluebox timelord) and installations using the packaged installer (which result in errors).

I've duplicated the problem across 3 bare metal devices as well as several VMs. Here are the CPU specs and relevant OS information for all:

Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz 2.59 GHz Edition: Windows 11 Pro Version: 23H2 Build: 22631.4037

Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz 3.90 GHz Edition: Windows 10 Pro Version: 22H2 Build: 19045.4780

AMD Ryzen Threadripper 2950X 16-Core Processor 3.50 GHz Edition: Windows 10 Pro Version: 22H2 Build: 19045.4780

If I can provide any additional information don't hesitate to ask.

wjblanke commented 2 weeks ago

Yes I can reproduce it here now. Hmmm

wjblanke commented 1 week ago

https://github.com/Chia-Network/chia-blockchain/pull/18544

wjblanke commented 1 week ago

https://github.com/Chia-Network/chia-blockchain/pull/18546

better

wjblanke commented 2 days ago

assigning to earle to look at killing the entire process tree