Without this, snatch currently buffers everything in memory and does many copy operations while extending the download buffer: chunk_vector.extend_from_slice(&bytes_buffer[..n]);
This implementations opens the output file, and shares this instance using an Arc<Mutex<File>> with all download threads.
Now my cpu usage is down to 10% with 16MiB/s, the kernel is doing all the work (50% cpu) - so now we have a very acceptable overhead.
This might be done nicer, but this is my first rust code at all :) - Tips are welcome.
Wow, thank you for this great improvement! 👍
The PR seems good for me (I will just update two/three little things).
Thank you again for your contribution :-)
Without this, snatch currently buffers everything in memory and does many copy operations while extending the download buffer:
chunk_vector.extend_from_slice(&bytes_buffer[..n]);
This implementations opens the output file, and shares this instance using an
Arc<Mutex<File>>
with all download threads.Now my cpu usage is down to 10% with 16MiB/s, the kernel is doing all the work (50% cpu) - so now we have a very acceptable overhead.
This might be done nicer, but this is my first rust code at all :) - Tips are welcome.