flow.js media streaming

[drzraf]

I would like to upload MediaStream (as captured directly from a camera), without intermediary storage.

Flow.js documentation sheds light on

flow.assignBrowse(document.getElementById('browseButton'));
flow.assignDrop(document.getElementById('dropTarget'));

as the prefered way to select media to uploads. It assumes input are binary blocks of known size (as probably desired to show completion percentage). (Discussed here) and underlying HTML5 File.

Is there any reason for flow.js to exclusively restrict itself to these instead of ReadableStreams?

Under the hood, it could be worked around using: (initFileFn initiates the MediaRecorder)

initFileFn: function(flowObj)
readFileFn: async function(flowObj, startByte, endByte, fileType, chunk) // To read chunks from the recorder

and looks similar to others attempts with opengpg.js.

Is it realistic or is there any hard blockers to anticipate from such an approach? A readFileFn function reading from a stream may have to await for it, thus async. It does not seem support ATM.

[command-tab]

I think I'm doing what you describe in a Vue app of mine. I capture from a webcam or other video device with the MediaRecorder API, then add the resulting file to Flow.js for upload.

To capture the MediaRecorder chunks, I do:

mediaRecorder.ondataavailable = e => this.blobs.push(e.data) // `this.blobs` is just an array

Then, in the mediaRecorder instance's onstop method, I do:

// Stop all streams
this.stream.getTracks().forEach(s => s.stop())

// Join the MediaRecorder chunks in a single Blob instance
const blob = new Blob(this.blobs)

// These two line are just for immediate playback in a <video> tag
const url = URL.createObjectURL(blob)
this.$refs.preview.src = url

// Create a File from the Blob, and add to Flow.js as if it were a user-selected file
const file = new File([blob], 'Recording.webm')
this.flow.addFile(file)

[drzraf]

I think there are two problems with this approach:

• It's not stream based: Upload starts when recording finishes (instead of when it starts)
• Memory: All the streams must hold in memory in order to create an immutable Blob (of known size)

Could flow.js chunking overcome these two limitations to get a true streaming experience?

[drzraf]

https://github.com/flowjs/flow.js/pull/304 will fix this. (It's already doable but involves the burden of non-trivial code being written by the user).