Slow Speed Issue, When download near to complete

erickutcher / httpdownloader

HTTP(S) download manager that uses input/output completion ports (IOCP).

https://erickutcher.github.io/#HTTP_Downloader

558 stars 62 forks source link

Slow Speed Issue, When download near to complete #172

Open shahwaliullah opened 3 years ago

shahwaliullah commented 3 years ago

I get good speed, when i choose 8 segment/connection, it shows 8/8 connection, when it go to near 95-96% download, speed drop down, for example, if speed was 500 kb/s, then when 95-96% download, completed, speed goes down to 40-50 kb/s, and it shows connection 1/8

erickutcher commented 3 years ago

That's normal behavior. The first 7 connections finished downloading their segment and only 1 connection remained. The speed of each connection cumulatively adds up. If there's only one connection going, then it'll only be as fast as that connection.

What you can do to speed it up again is stop the download, and then start it. The remaining data will be split between the number of connections you supplied and you should get faster speeds.

JNavas2 commented 3 years ago

@erickutcher Yes, that can be an effective manual workaround, but it would be better to automatically reallocate connections as they complete. Please consider adding that, at least as an option. Thank you.

erickutcher commented 3 years ago

I'll have to give it some thought. I can see some potential issues having it automated so I'd need to think of an appropriate way to handle it.

JNavas2 commented 3 years ago

@erickutcher Understood. Thank you!

erickutcher commented 2 years ago

A new version is up that implements the reallocation feature. It's "Reallocate parts to maximize connections" under the Connection Options.

JNavas2 commented 2 years ago

A new version is up that implements the reallocation feature. It's "Reallocate parts to maximize connections" under the Connection Options.

Wow, you rock! It's almost perfect. The only problem I see is on completion. It hit 100% on an 8-part download, then stalled, very slowly closing all the remaining connections one by one. Screen shot shortly after it hit 100%:

JNavas2 commented 2 years ago

Video of stalling at completion: https://youtu.be/KMRJ7urC97A

erickutcher commented 2 years ago

Does it consistently do this? I've tried downloading a bunch of different Linux isos to see if I can reproduce it, but it's not happening for me.

Do you have a URL that I can download to test this?

JNavas2 commented 2 years ago

Does it consistently do this? I've tried downloading a bunch of different Linux isos to see if I can reproduce it, but it's not happening for me.

It's consistent on a particular hosting site. I have not tried others.

Do you have a URL that I can download to test this?

Send me an email (link on my website), and I tell you how to reproduce it.

erickutcher commented 2 years ago

I think I may know what's going on. If the site is responding with a "Connection: close" header value, then the program is over-downloading the parts that are split. It all works properly if the site accepts a keep-alive connection...which most do. This won't happen with FTP or SFTP connections though. It's just an HTTP thing.

I'll try to get a new version up in a couple days to fix this.

JNavas2 commented 2 years ago

I'll try to get a new version up in a couple days to fix this.

Thank you!

JNavas2 commented 2 years ago

I just tested an HTTP download from Google Drive, and it completed properly, so this must be something unique to particular websites.

erickutcher commented 2 years ago

Give this beta a try. I found a simple fix for the problem I described and I think it should resolve what you've been experiencing. HTTP_Downloader_64_1.0.4.7_Beta.zip

JNavas2 commented 2 years ago

Perfect! I will show my appreciation with a donation.

erickutcher commented 2 years ago

Thank you. I do appreciate that. I'm glad it's working. I'll have a new release up some time tomorrow.

erickutcher commented 2 years ago

A new version is up. Thanks again for the donation.

JNavas2 commented 2 years ago

I've run into an apparent edge case where reallocation fails. With 3 downloads, I stopped all 3 so I could reorder them (absent queuing), then started #2, #3, and #1, in that order. Some time later I checked back, and found #2 stuck in only 4 out of 8 parts active. See below. I stopped #1 and #2, then started #2 and #1 in that order, and #2 then completed with 8 active until near the end. It's not a huge deal, but let me know if I can help in any way.

erickutcher commented 2 years ago

I'm thinking that maybe the remote server closed those connections before their parts could complete. The reallocation only happens when a part fully downloads. By default the program doesn't retry incomplete parts, but you can change that in the Connection options.

JNavas2 commented 2 years ago

I have that option on. See below. Maybe it has something to do with Stopping and then Starting the download?

erickutcher commented 2 years ago

It's most likely a connection issue with the remote server. I can get it to happen if I use a program like CurrPorts to kill some of the connections, but stopping and starting the downloads don't seem to have any effect on my end. It all seems to work as it should. I'd probably just increase the retry count to see if it stabilizes things.

JNavas2 commented 2 years ago

Thank you. As I said, it's not a huge deal. What you've already done has made the huge difference for me.