search

derniercri / snatch

A simple, fast and interruptable download accelerator, written in Rust
MIT License
676 stars 37 forks source link

What is the optimal thread count for a 64 bit Macbook Pro? #45

Closed MeetThePatel closed 3 months ago

MeetThePatel commented 7 years ago

I tried to do a thread count of 4, and then a thread count of 64, and to the naked eye, for a small file, it seemed indistinguishable. What is the optimal thread count?

k0pernicus commented 7 years ago

Hi @MeetThePatel, In the devel branch, we ensure that Snatch runs at least one thread per (physical) core automatically, but this is not the optimal thread count. Currently, we use the num-cpus crate to obtain informations about your machine (this crate parse the /proc/cpuinfo file to obtain those), but this crate does not give us the number of threads per core we can use. We plan actually to have a better automatic behaviour to get a better optimal thread count for a machine, parsing the lscpu command output maybe...

Currently, I think that you can use the default (and automatic) behaviour of Snatch to download a big file - maybe that you can multiply the number of threads (to see it, please to add the -d flag) by two, but if you increase again the multiplier it can have some side effects on your system and on Snatch

lambdaupb commented 7 years ago

The thread count has little to do with the computer/CPU you use, only with the remote server you are downloading from and your internet connection.

It makes little sense to adapt the thread count to /proc/cpuinfo since the threads will be blocking 90% of the time waiting for new data to come from the connection socket.

This type of "download acceleration" only really helps with download servers that limit the download speed per connection, or overloaded servers that give every connection a fraction of the available bandwidth - by increasing your connections you represent more connections and steal more bandwidth in comparison to other clients that only use one.