Standalone hosting (ie, without relying on jimmywarting.github.io)?

[mcclure]

As described in other issues I'm using StreamSaver installed from npm in a test app which I'm building with webpack and serving from npm http-server running on localhost. The app is working. But I notice:

• When I use the streamsaver feature the Network panel in the Chrome shows several connections to github.io.
• There is a section at the end of the documentation explaining, in part:

  StreamSaver creates a own man in the middle that installs the service worker in a secure context hosted on github static pages. either from a iframe (in secure context) or a new popup if your page is insecure.

That makes sense. However, in the particular app I'm writing, for both philosophical and practical reasons (for example, what if I wanted to support an offline mode?) I would prefer if possible for my app to be totally self-contained. Assume I have already ensured my app is running in the "Secure Context".

Is there a way to serve the MITM bits locally, either with the mitm bits packed in with webpack, or served from a separate file?

Would it be possible for you to document the process for doing this, even if it's not formally supported by the software? The documentation implies there are steps needed for the github.io stuff to work besides just being served from https, but doesn't say what they are.

[jimmywarting]

You are right, the documentation could benefit from how to set up a own hosting env i did the "man in the middle" so ppl could just include it from npm or sandboxed enviorment like jsfiddle and have support for insecure sites as well without having to deal with setting up service worker.

---

One thing I would like to have in those scenarios is a StreamSaver without a "man in the middle" and sending the postMessages directly to service worker instead but that would require a bit of refactoring.

I guess the easiest way atm is to host the mitm.html and service worker on your own domain and point the url to your own place

import streamSaver from 'streamsaver'
streamSaver.mitm = 'https://example.com/mitm.html'

// use StreamSaver as usual

---

Would also like to point out that StreamSaver do lack backpressure all together where it dose not notify back to the main thread that it's ready to accept more data. it lacks the concept of a bucket. It means you have no idea if the user canceled the download or not, so you will keep on sending chunks of data, and doing writer.write(chunk) resolves immediately, you have no clue if it's done writing. All that is fixed automatically if the browser supports Transferable ReadableStream

I also believe this StreamSaver, FileSaver and other alike will be superseded later by the Native Filesystem API. So at some point in the feature i will deprecate this but not yet for a wile. For this reason i have also built a Native FileSystem adapter that is capable to do most thing that the native file system can do but you can decide for your self where the data should be read/written to. It includes

• IndexedDB
• blinks sandboxed filesystem
• cache storage
• memory
• native file system (+its sandbox)
• And also to service worker as a way to download.
  This adapter borrows some ideas from StreamSaver to be able to write directly to the disc too. The differences is:
  • It communicates directly to a self hosted service worker
  • Handles backpressure correctly. So you can detect more easier when the pipeline interrupts, gets paused or aborted. if the chunk you write writer.write(chunk).then(...) is completed or not

My adapter is a bit like a polyfill in itself and you can write more than just Uint8Arrays. including blob, string, buffers or any typed array. I have also made it somewhat optional to use service worker or not so it can write to a memory and later save a hole blob which you are required to do in Safari.

Maybe you would like to try my FilsSystem adapter instead?

[EtienneBruines]

@mcclure I have been able to incorporate the MITM parts in my codebase:

• Host the StreamSaver.html file somewhere. As part of the rollup/webpack you can copy it over to your dist/ folder.
• Copy the contents of the sw.js into my own service worker. This meant adding some code in the pre-existing fetch event listener, as well as the functions listed.
• Be sure to pass the correct path to your service worker in the registration in StreamSaver.html.

@jimmywarting I can imagine, for compatibility-reasons, it being nice if the service-worker was refactored slightly? Having a separate js file containing the functions, allowing it to be imported by someone else's service worker as well.

• isStreamSaverFetch
• handleStreamSaverFetch
• isStreamSaverMessage
• handleStreamSaverMessage

I am not sure if one could potentially just call StreamSaver.initialize() which would add event listeners for fetch and message... I don't know how browsers handle multiple event listeners.

I would imagine the code in someone else's service worker to look something like:

import { isStreamSaverFetch, handleStreamSaverFetch, isStreamSaverMessage, handleStreamSaverMessage } from 'stream-saver'

self.addEventListener('fetch', (event) => {
  if (isStreamSaverFetch(event)) { // Would also handle /ping, but we might want to prefix stuff with /stream-saver or something, to make it easier to distinguish
    handleStreamSaverFetch(event)
    return
  }

  // ... other contents of fetch handler
}

self.addEventListener('message', (event) => {
  if (isStreamSaverMessage(event)) {
    handleStreamSaverMessage(event)
    return
  }

  // ... other contents of message handler
}

That way, if the logic changes (e.g. which headers to add, how to name files, etc.) -- it would stay up-to-date with the StreamSaver code.

---

Bonus-idea: we could even serve the StreamSaver.html page as part of the service worker code. That way, just registering the service-worker code would suffice. (And setting the MITM-URL in the actual code, but that's to be expected)

Before anyone puts in any effort into doing this, I thought I'd ask your opinion first? :smile:

---

Alternatively, handling it and returning true, or false if not handled:

import { handleStreamSaverFetch, handleStreamSaverMessage } from 'stream-saver'

self.addEventListener('fetch', (event) => {
  if (handleStreamSaverFetch(event)) { // Would also handle /ping, but we might want to prefix stuff with /stream-saver or something, to make it easier to distinguish
    return
  }

  // ... other contents of fetch handler
}

self.addEventListener('message', (event) => {
  if (handleStreamSaverMessage(event)) {
    return
  }

  // ... other contents of message handler
}

[stefanfis]

Additionally to @jimmywarting's instructions I had to host sw.js alongside the mitm.html file on my own server.

Also, my Nginx installation added a X-Frame-Options: DENY header to every file served, so I had to remove this as well:

location ~ /mitm.html { 
    add_header X-Frame-Options ""; 
}