Use Channels rather than BlockingCollection for block processing

NethermindEth / nethermind

A robust execution client for Ethereum node operators.

https://nethermind.io/nethermind-client

GNU General Public License v3.0

1.28k stars 445 forks source link

Use Channels rather than BlockingCollection for block processing #7790

Open benaadams opened 4 days ago

benaadams commented 4 days ago

Changes

Wrapping a ConcurrentQueue in a BlockingCollection burns a ton of CPU spinning; whereas its a pattern ideally suited to Channels; and using true async for rather than TaskCompletionSource for async is lower overhead.
Faster full sync

Types of changes

What types of changes does your code introduce?

[x] Optimization
[x] Refactoring

Testing

Requires testing

[x] No

benaadams commented 4 days ago

Recovery thread should also have boosted priority.

Is start of block; is either not doing anything or is doing catch up when main thread is processing and then its competing with the Parallel.For s in main processing

My main concern is that before recovery and processing had dedicated threads and now they will compete on thread pool.

Can't use async on dedicated thread or loose the thread on first await. Don't think is hugely significant; still boost main processing and Recovery doesn't out compete all the parallel processing of main thread.