search

marklieberman / downloadstar

Download all items in a webpage that match a pattern
GNU General Public License v3.0
90 stars 16 forks source link

Download with authentication #55

Closed CasualX closed 5 years ago

CasualX commented 6 years ago

Hello,

I am trying to mass download documents from a CMS where the built in download tools suck, so I am trying to solve my problem with Download Star (after finding out that DownThemAll! is no longer working).

The page I am trying to work with contains a list of documents and for each entry contains (among other things) a download link. All the download links contain the text tool_action=downloader.

Filtering these links with Download Star works just fine however after download I am left with contents of a .html file which when opened contain a login form. It is as if the authentication cookies do not transfer to the Download Star downloads.

Am I missing something? Is this use case supported by Download Star?

Thanks.

marklieberman commented 6 years ago

Cookies are not added to the download requests at this time, so downloads often don't work for sites with auth checks.

The problem is that the web extension API download.download() doesn't automatically add cookies for a domain to the download requests. Although you can manually set headers on the download, you cannot set the Cookie header because it is on the forbidden headers list.

When I have some more free time I will investigate further to see if there are any work-arounds.

CasualX commented 6 years ago

The limitation is unfortunate, thanks for the fast response!

deant commented 5 years ago

I came here to report the same bug. I installed downloadstar to help with lots of downloads on a specific site, and ran into this exact same problem. If it's at all possible to fix or work around this issue, that should be a high priority!

marklieberman commented 5 years ago

Closing in favour of #72 that has more information.