search

erickutcher / httpdownloader

HTTP(S) download manager that uses input/output completion ports (IOCP).
https://erickutcher.github.io/#HTTP_Downloader
571 stars 62 forks source link

Saved username & password in "Login Manager", doesn't work #113

Closed rayman3003 closed 4 years ago

rayman3003 commented 4 years ago

Downloading from this website, needs a free registration. After registration, its possible to download from this website:

https://battleofthebits.org/

Here is an example link:

https://battleofthebits.org/player/EntryDonload/33140/BotB%2033140%20DCT%20Master%20-%20Collective%20Minds.mp3

I can download this file by browser (Firefox), without problem.

So I put my username & password in "Login Manager" of "HTTP Downloader", But whenever I add a url from the above website to "HTTP Downloader", the username & password sections r always blank:

Screenshot:

2020-01-23 12_18_42-Window

Also, Even If I type my username & password in "add url(s)" windows, "HTTP Downloader" can't download the file either!

erickutcher commented 4 years ago

Most likely there's some cookie values that are needed in addition to the authentication information. I'll look at this this weekend.

rayman3003 commented 4 years ago

Most likely there's some cookie values that are needed in addition to the authentication information. I'll look at this this weekend.

Thank u for ur time.

erickutcher commented 4 years ago

You don't need to use any authentication in the program to download those files, however, you will need the correct cookie values and referer information. The best way to download from that site is to use the browser extension. The extension will send all the required information (cookies and headers) to HTTP Download in order to download properly.

rayman3003 commented 4 years ago

You don't need to use any authentication in the program to download those files, however, you will need the correct cookie values and referer information. The best way to download from that site is to use the browser extension. The extension will send all the required information (cookies and headers) to HTTP Download in order to download properly.

It is not 1 or 2 file. Its more than 1000 files. I don't think its possible to use extension to send 1000 files (at the same time) to the downloader. Or is it possible?

erickutcher commented 4 years ago

If all of the URL(s) you want to download are on a single page, then you can right click on the page (when the extension is installed) and select "HTTP Downloader > Download All Media..." or "HTTP Downloader > Download All Links..." and filter out the list of URL(s) that you want.

rayman3003 commented 4 years ago

If all of the URL(s) you want to download are on a single page, then you can right click on the page (when the extension is installed) and select "HTTP Downloader > Download All Media..." or "HTTP Downloader > Download All Links..." and filter out the list of URL(s) that you want.

Thank u, But it seems I have a problem for the extension here:

This is the first time I'm using the extension.

I set the server in both "HTTP Downloader" & the extension to: 127.0.0.1:80

But When I try to add links from extension to "HTTP Downloader", It shows the following error:

"An error occurred while sending the download request"

Screenshot:

https://i.postimg.cc/Z51Bdm6V/0000.jpg

erickutcher commented 4 years ago

The server setting in the extension should be: http://127.0.0.1:80

rayman3003 commented 4 years ago

The server setting in the extension should be: http://127.0.0.1:80

Changed it to "http://127.0.0.1:80" as u said. But now its showing the following error:

https://i.postimg.cc/rFcWV0Bc/000.jpg

erickutcher commented 4 years ago

You don't happen to have something else running on your system that's listening on port 80?

Try changing the port in the extension and the program to something different. For example, port 41234

rayman3003 commented 4 years ago

port 41234

OK, I changed the port & now the extension can communicate with downloader. :D

But still a problem:

For example I try to download files in this link (*.sid extension files):

https://battleofthebits.org/browser/EntryByFormat/sid/

When I click on one of "donload" buttons & choose "download link" with extension, It can download that single file successfully. (*.sid extension files)

But If I choose to mass download all the sid files on that page (by "download all links" of extension), Downloader only download .html files or .sid files that r corrupted (with 152 bytes sizes).

Single download works, but mass download not working!

erickutcher commented 4 years ago

That's a cookie issue. When you download from a single link, any cookies associated with that host will be included in the Add URL(s) window. However, if you select one of the "Download All..." menu options, then the cookies are not included. The reason for that is that there might be other hosts in the URL list that shouldn't receive another host's cookies. It's a security thing.

Here's what you can do with that particular site since downloading from it is dependent on cookies.

Right click on one of those download links and select "HTTP Downloader > Download Link...". In the Add URL(s) window put a check in "Advanced options". Copy the cookies in the Cookies textbox and then close the Add URL(s) window.

Next, right click on the page and select "HTTP Downloader > Download All Links...". Select the "Adavanced options" in the Add URL(s) window and then paste the cookies in the Cookie textbox. You can also filter the .sid files using the following regular expression: \.sid$

That should supply HTTP Downloader with all it needs to download those files properly.

rayman3003 commented 4 years ago

Right click on one of those download links and select "HTTP Downloader > Download Link...". In the Add URL(s) window put a check in "Advanced options". Copy the cookies in the Cookies textbox and then close the Add URL(s) window.

Next, right click on the page and select "HTTP Downloader > Download All Links...". Select the "Adavanced options" in the Add URL(s) window and then paste the cookies in the Cookie textbox. You can also filter the .sid files using the following regular expression: \.sid$

That should supply HTTP Downloader with all it needs to download those files properly.

It worked.

Thank u very much. :)

erickutcher commented 4 years ago

I've modified the browser extensions a bit so that they can get the cookies when selecting one of the "Download All..." menu items. It only works if all of the links that are gotten are from the same domain. To force it to only grab URLs from the same domain as the page you're on, you'll have to hold Ctrl before clicking the menu items.

That'll be available in the next release.

rayman3003 commented 4 years ago

I've modified the browser extensions a bit so that they can get the cookies when selecting one of the "Download All..." menu items. It only works if all of the links that are gotten are from the same domain. To force it to only grab URLs from the same domain as the page your own, you'll have to hold Ctrl before clicking the menu items.

That'll be available in the next release.

Its awesome. Thank u.

erickutcher commented 4 years ago

New versions of the extensions are up. If you select one of the "Download All..." menu options and all of the URLs have the same base domain, then it'll load all the cookies that are associated with it. To force it to grab only URLs from the domain of the current page, you'll want to hold Ctrl while clicking one of those "Download All.." menu options. This only works in the Firefox extension. For the Chrome extension, there's an extra menu option ("From Current Domain") that'll let you toggle that feature on and off.