Chia-Network / chia-blockchain

Chia blockchain python implementation (full node, farmer, harvester, timelord, and wallet)
Apache License 2.0
10.82k stars 2.03k forks source link

[BUG] 2 out of 8 parallel plot jobs fail silently, UI continues to report Plotting #1361

Closed centrd closed 3 years ago

centrd commented 3 years ago

Describe the bug 2 out of 8 parallel plot jobs fail silently, UI continues to report Plotting. Traceback can be seen in log, as well as a parse error.

What I did:

  1. Installed Chia 1.0rc9
  2. Upgraded to Chia 1.0.0
  3. Started 8 parallel plot jobs

Result: 2 jobs (the bottom two on the plotter list) failed silently, while the UI continue to claim "Plotting". Error report can be seen when clicking on view log.

Expected behavior If a plotting job fails, the UI should reflect that. Optionally, a notification would be nice.

Log

23:41:41.628 src.plotting.create_plots : INFO  Adding directory F:\ to harvester for farming [6800] Failed to execute script chia Traceback (most recent call last): File "src\cmds\chia.py", line 63, in File "src\cmds\chia.py", line 59, in main File "click\core.py", line 1026, in call File "click\core.py", line 956, in main File "click\core.py", line 1518, in invoke File "click\core.py", line 1518, in invoke File "click\core.py", line 1280, in invoke File "click\core.py", line 711, in invoke File "click\decorators.py", line 22, in new_func File "src\cmds\plots.py", line 134, in create_cmd File "src\plotting\create_plots.py", line 158, in create_plots File "src\plotting\plot_tools.py", line 107, in add_plot_directory File "src\util\config.py", line 55, in load_config File "yaml__init__.py", line 162, in safe_load File "yaml__init__.py", line 114, in load File "yaml\constructor.py", line 49, in get_single_data File "yaml\composer.py", line 36, in get_single_node File "yaml\composer.py", line 55, in compose_document File "yaml\composer.py", line 84, in compose_node File "yaml\composer.py", line 133, in compose_mapping_node File "yaml\composer.py", line 84, in compose_node File "yaml\composer.py", line 127, in compose_mapping_node File "yaml\parser.py", line 98, in check_event File "yaml\parser.py", line 428, in parse_block_mapping_key File "yaml\scanner.py", line 115, in check_token File "yaml\scanner.py", line 152, in need_more_tokens File "yaml\scanner.py", line 292, in stale_possible_simple_keys yaml.scanner.ScannerError: while scanning a simple key in "C:\Users\Administrator.chia\mainnet\config\config.yaml", line 244, column 1 could not find expected ':' in "C:\Users\Administrator.chia\mainnet\config\config.yaml", line 245, column 1

Desktop (please complete the following information):

centrd commented 3 years ago

Update Came back to the plotter this evening, and Chia wasn't running. Checked Event Viewer, and found an app crash:

Faulting application name: Chia.exe, version: 1.0.0.0, time stamp: 0x60301e4e Faulting module name: KERNELBASE.dll, version: 10.0.17763.1790, time stamp: 0xc97af40a Exception code: 0xe0000008 Fault offset: 0x0000000000039689 Faulting process id: 0xcac Faulting application start time: 0x01d71ba8351c2600 Faulting application path: C:\Users\Administrator\AppData\Local\chia-blockchain\app-1.0.0\Chia.exe Faulting module path: C:\Windows\System32\KERNELBASE.dll Report Id: 951a7032-b67e-44d7-a6c3-ddeedb0ee237 Faulting package full name: Faulting package-relative application ID:

Five plots completed overnight.

wjblanke commented 3 years ago

yaml.scanner.ScannerError: while scanning a simple key in "C:\Users\Administrator.chia\mainnet\config\config.yaml", line 244, column 1 could not find expected ':' in "C:\Users\Administrator.chia\mainnet\config\config.yaml", line 245, column 1

Can you show us whats in your config.yaml around lines 244/245?

centrd commented 3 years ago

Also noteworthy: .chia is a folder under Administrator. There is a \ missing in the path for some reason.

243 target_peer_count: 5 244 testing: false 245 wallet_peers_path: wallet/db/wallet_peers.sqlite 246 (line 246 is blank)

Screenshot: Fzf7foR

centrd commented 3 years ago

Another crash of the UI 8:10:51AM local time. Started plotting jobs last night. Lots of disk activity still happening, but I'm not sure if it's plotting stuff that will completed and usable.

When I re-opened the UI, it started showing database locked errors after a couple of minutes running.

Faulting application name: Chia.exe, version: 1.0.0.0, time stamp: 0x60301e4e Faulting module name: KERNELBASE.dll, version: 10.0.17763.1790, time stamp: 0xc97af40a Exception code: 0xe0000008 Fault offset: 0x0000000000039689 Faulting process id: 0x1d38 Faulting application start time: 0x01d71c5168a45deb Faulting application path: C:\Users\Administrator\AppData\Local\chia-blockchain\app-1.0.0\Chia.exe Faulting module path: C:\Windows\System32\KERNELBASE.dll Report Id: 0a25f2c5-2641-40bf-a381-32507e90935c Faulting package full name: Faulting package-relative application ID:

centrd commented 3 years ago

Crashed again, this time around 2.23 AM. More plot jobs have been added since the last crash. Lots of disk IO still happening, so looks like it's continuing to plot, but the UI has crashed. Same exception code and fault offset.

centrd commented 3 years ago

Crashes are daily. Plotting continues in the background after UI crash.

centrd commented 3 years ago

I've given up and switched to using the terminal + scripts. Leaving the issue open.

gmjgh commented 3 years ago

Same here. Got this error twice. It seems that nothing gets broken but it is quite uncomfortable

github-actions[bot] commented 3 years ago

This issue has been flagged as stale as there has been no activity on it in 14 days. If this issue is still affecting you and in need of review, please update it to keep it open.

github-actions[bot] commented 3 years ago

This issue was automatically closed because it has been flagged as stale and subsequently passed 7 days with no further activity.