Open ANogin opened 1 month ago
No longer sure this would make things faster rather than slower, marking as draft for now.
I did a timing test of a gather of 2x (create_root_resource_from_file
+ unpack_recursively(do_not_unpack=[Elf])
+ auto_run_recursively(all_identifiers=True)
) on TinyCore iso 14 and 15 respectively. Here are the results
feature/aiofiles 114.38s wall 80.99s CPU 873.3ms async select 32.52s blocked
feature/aiofiles 113.00s wall 82.09s CPU 903.2ms async select 30.01s blocked
feature/aiofiles 120.63s wall 86.49s CPU 1.01s async select 33.13s blocked
feature/aiofiles 120.12s wall 85.63s CPU 879.3ms async select 33.61s blocked
feature/aiofiles 118.97s wall 84.46s CPU 876.1ms async select 33.63s blocked
master f3e583c 108.79s wall 76.34s CPU 146.6ms async select 32.31s blocked
master f3e583c 111.36s wall 77.90s CPU 151.1ms async select 33.31s blocked
master f3e583c 109.05s wall 76.93s CPU 146.9ms async select 31.98s blocked
master f3e583c 113.03s wall 78.98s CPU 144.8ms async select 33.91s blocked
master f3e583c 112.77s wall 79.39s CPU 130.9ms async select 33.25s blocked
where I used the approach from https://gist.github.com/vxgmichel/620eb3a02d97d3da9dacdc508a5d5321 to break the non-CPU time into “select” vs “blocked IO”.
With #472 merged into master and into this branch:
master 27.35s wall 26.99s CPU 124.0ms async select 244.7ms blocked
master 26.97s wall 26.55s CPU 125.3ms async select 293.7ms blocked
master 26.90s wall 26.52s CPU 120.5ms async select 265.8ms blocked
master 27.68s wall 27.29s CPU 123.3ms async select 267.2ms blocked
master 26.91s wall 26.51s CPU 131.1ms async select 267.9ms blocked
feature/aiofiles 30.29s wall 31.06s CPU 951.7ms async select 0.0ms blocked
feature/aiofiles 29.69s wall 30.39s CPU 1.02s async select 0.0ms blocked
feature/aiofiles 29.87s wall 30.59s CPU 910.6ms async select 0.0ms blocked
feature/aiofiles 29.66s wall 30.44s CPU 948.8ms async select 0.0ms blocked
feature/aiofiles 29.65s wall 30.45s CPU 922.8ms async select 0.0ms blocked
(the "0ms blocked" is an underapproximation)
One sentence summary of this PR (This should go in the CHANGELOG!) Use
aiofiles
for filesystem IOLink to Related Issue(s)
Please describe the changes in your request. Replace blocking IO for file read/write with async io using
aiofiles
Anyone you think should look at this, specifically? Not sure