Use a `js-ipfs` instance instead of using the public gateways

[fsdiogo]

At the moment if you want to add files you need a local daemon running. To get files we're using the public gateways.

The reasoning behind this was to make the files sharing flow easier for everyone: one who wants to add files to IPFS would have a bit of understanding of IPFS (that's why he chose to share files via IPFS) so it would use a local daemon to do so. But he may want to transfer those files to a person that doesn't even know about IPFS, so the end user (the user who'll get the files) just needs to open the browser and can download the files with no extra steps.

That is only partially true, if a user heard about IPFS and wants to try to share something via IPFS, he'll need to install it, run a local daemon and then add a file through the app. It's a lot of steps for someone who just wants to try it out, we should try to make it as smooth and simple as possible to grab his attention. If everything goes smoothly, the higher the chances that the user starts to dig into IPFS and then install it and run a local daemon, etc.

So, why not make it easier for both parties? We can use the ipfs-redux-bundle to choose the best available option (Companion, IPFS API, ...), if there is none we just fire up a jsipfs instance, for both the user who's adding files and getting them.

TL;DR: let's find the best available option through ipfs-redux-bundle and use an instance of jsipfs as fallback instead of the public gateways.

Makes sense @ipfs-shipyard/gui?

[lidel]

I'm :+1: for adding ipfs-redux-bundle: it already supports detection of webextension context, so it will make embedding the app with ipfs-companion a lot easier (we want to replace current upload screen with share app).

Something to be aware of when enabling fallback to js-ipfs on a regular website (without companion):

It may introduce additional constraints in form of browser storage limits (eg. user uploads 3GB file but it fails in the middle of upload due to js-datastore-level hitting storage quota). We should have error handling in place when that happens and display actionable feedback to the user (eg. "if you want to share files bigger than X install ipfs-desktop/ipfs-companion")

Some related resources:

• https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Browser_storage_limits_and_eviction_criteria
• Limits in 2014: https://www.html5rocks.com/en/tutorials/offline/quota-research/
  • Browser Storage Abuser: https://demo.agektmr.com/storage/

[olizilla]

Great! The main goal of this change is to not rely directly on the public http-to-ipfs gateway directly for the downloader who wants to get the file. Using the best local IPFS available is a better demonstration of the technology, and should reduce the burden on the gateways a little.

For the uploader, it makes sense to do them same thing; use the best IPFS available, but this is a secondary goal.

One minor issue is that if you choose a custom API addr in webui, we need to get that info across to share.ipfs.io but localStorage is silo'd by origin. We could use query strings to pass custom api address, or use an iframe to ipfs.io and set shared info there. Let's get the app working with ipfs-redux-bundle and open an issue for communicating custom api addresses as a seperate issue.

[fsdiogo]

After having a chat with @olizilla we uncovered some pain points about not using the public gateways, and some solutions to that as well.

Here goes some of the actionable items we have to handle:

• [x] Provide download progress feedback

  • When downloading files we were using XMLHttpRequest to request the public gateway and using the onprogress function to the show the download progress;
  • Now we'll use an instance of ipfs so we will get the files from a call to ipfs.files.get, which doesn't give us any kind of progress. A solution to this is to use streams and manually calculate the download progress;

• [x] Handle large files

  • As @lidel mentioned above, we'll have to deal with the browser constraints around large files;
  • When fetching the files list with ipfs.ls we can check the size of large files, signal them and warn the user that he won't be able to download them in the app. We'll point him to install IPFS Desktop so he can circumvent that problem;

• [x] Download multiple files

  • The public gateway gives us the ability to wrap multiple files into an archive
  • The ipfs api doesn't allow us to do that, so for now we'll keep the download all functionality, downloading every file individually