Download attached files in the Browser's context

[reox]

I often have the problem, that articles that are imported via the browser extension have linked PDFs, which I can not download via JabRef itself (for example via https://github.com/JabRef/jabref/issues/5662 or by browsing to the PDF and then starting the browser extension). That might be because you have to be logged in into a website or due to other reasons. Some pages return an error 403 (sciencedirect, researchgate), others return 503 (wiley - maybe that one is a different problem though).

However, if I open the link in the browser, I get the PDF immediately. Thus, it would be convenient to be able to download the PDF directly in the JabRef extension (so that it has the browser context and cookies (is that possible?)) and then send the downloaded PDF and metadata into Jabref.

edit: I quickly checked what's going on with wiley. The problem seems to be cloudflare. When I use wget or curl to access the PDF, I get immediately a 503 error and a html page which says at the bottom:

DDoS protection by <a rel="noopener noreferrer" href="https://www.cloudflare.com/5xx-error-landing/" target="_blank">Cloudflare</a>

[tobiasdiez]

That's a good idea, and would be possible to implement using https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/downloads/download. We should also add an option that enables this in-the-browser download behavior, as some people may not want to download files at all. PR is very welcome.

[ror3d]

I've been thinking a bit on how this could be solved. Several points come to mind:

• As opposed to Zotero, JabRef's plugin doesn't directly add entries to the library, but sends them to be imported through the import dialog, because the communication between the two applications (jabref and plugin) is minimal. This makes asynchronous downloading of the attached files more complicated. Either:
  • the files have to be downloaded prior to opening JabRef so it already has the file information, or
  • some way needs to be devised to send the updated information to JabRef
  • then the possibility of adding a file but cancelling the import before it is downloaded and sent to JabRef, and other similar edge cases, have to also be considered.
• Having a more direct connection between the plugin and JabRef using connectNative sockets instead of sendNativeMessage might allow for easier control of these cases (the entry could be directly created and then the files attached to it once they are downloaded). This would require a bunch of work in different ways than the previous solution.
• Trying to download files from JabRef after-the-fact would still have the same issues with not having a proper context to download past paywalls.
  • A more direct connection through connectNative might allow for downloading files after-the-fact through paywalls more easily from inside JabRef
  • An embedded browser in JabRef could be implemented instead to be able to access such files. This could solve both problems through a different approach. It might be quite tricky to implement so it works well.

[tobiasdiez]

* Having a more direct connection between the plugin and JabRef using connectNative sockets instead of sendNativeMessage might allow for easier control of these cases (the entry could be directly created and then the files attached to it once they are downloaded). This would require a bunch of work in different ways than the previous solution.

This would be my preference. Either using connectNative or using a websocket connection (i.e. implement a websocket server in the main jabref application, similar in spirit to https://github.com/karlicoss/grasp/blob/master/extension/src/js/background.js). Not sure what is easier to implement and more stable. I sadly don't have the time to work on it myself, but would of course accept PRs and help with the implementation.

Such a connection would be also handy to implement other features. For example, https://github.com/JabRef/JabRef-Browser-Extension/issues/32.