search

avamsi / Flash

Flash is a highly parallel downloading client with support for multiple source IP addresses / network adapters
12 stars 3 forks source link

Speed comparison aria2c vs flash? #1

Open kootenpv opened 8 years ago

kootenpv commented 8 years ago

Hey, I'm wondering if you're sure you need different IP addresses... could you test out your tool and see if you can get a faster download speed in comparison to this tool (uses multiple threads):

brew install aria2 # OSX
apt-get install aria2 # ubuntu

You can then run the following command to use 16 threads:

aria2c -x 16 -s 16 http://some_url/some_big_file.tar
avamsi commented 8 years ago

Hey @kootenpv!

I run Windows :P but I already have aria2 installed. I used to use aria2 before Flash.

The problem with aria2 is, default max connections to a server is 16. If you need more than that, you have to change it in the source and compile it yourself -- the process wasn't straightforward, at least at the time I was not able to do it.

I normally use 40-50 IP addresses so 16 was not enough for me, so I wrote my own downloader "Flash" I max out on my network ethernet speed with Flash -- so it can't get any faster than that. As downloaders are almost always network bound, Python is not a problem either.

PS. Flash uses multiple threads too. As of now, it uses 8*NUMBER_OF_IP_ADDRESSES threads. At a given time only NUMBER_OF_IP_ADDRESSES threads are actually downloading data but 8*NUMBER_OF_IP_ADDRESSES threads are created at the beginning itself.

kootenpv commented 8 years ago

Cool, very nice backstory! I just read up on IP, it's a clever addition over aria2c (even though in my case multiple threads were fine compared to requiring multiple IPs). If I were you I would add some kind of "official" benchmark to your readme, that'll help convince people it is very serious :)

EDIT: And indeed, I missed that it is on Windows. It would probably be nice to make it work on all OSes. How are you creating multiple IPs?

avamsi commented 8 years ago

How are you creating multiple IPs?

@kootenpv, I don't understand the question.

On my college network, downloaded speed is limited per IP address but many IP addresses are unused. So, you can just add multiple IP addresses to your network adapter and then download parallelly from all of them.

Say, the speed limit is 3 Mbps per IP addresses, and I use 30 - 40 IP addresses, I max out at ~100 Mbps. If you are asking me how I add multiple IP addresses to network adapter, I wrote a thin Python wrapper around (Windows) netsh command.