Neptune-Crypto / neptune-core

anonymous peer-to-peer cash
Apache License 2.0
23 stars 7 forks source link

Make File Operations async. (ArchivalState, etc) #75

Closed dan-da closed 3 months ago

dan-da commented 7 months ago

Related to #74.

Presently many, if not all, file operations use synchronous APIs, called from async functions. For example async fn get_latest_block() eventually calls get_block_from_record which opens a file and reads it in via (unsafe) mmap calls.

These sync operations will block the tokio executor which can result in poor performance/concurrency. So in general, it should be more performant to convert the file ops to async counterparts provided by tokio, or at least wrap them with spawn_blocking.

note: I'm unsure about making the mmap calls async; an area for research.

dan-da commented 5 months ago

The only remaining blocking I/O that I'm aware of is in twenty-first storage. To fix that, we need to add a tokio dep in twenty-first, or move storage out of twenty-first somehow.

It's also possible there is blocking I/O in triton_vm or tasm-lib, but I'm hoping not.... ;-) Another consideration there is that compute-heavy functions should also be made into tokio tasks to avoid concurrency problems. I haven't investigated if we are doing that or not, eg when exec'ing VM code.

dan-da commented 3 months ago

closed by #124