search

publiclab / image-sequencer

A pure JavaScript sequential image processing system, inspired by storyboards
https://sequencer.publiclab.org
GNU General Public License v3.0
110 stars 208 forks source link

Use blobs instead of dataurls to pass data between modules #1847

Open jywarren opened 3 years ago

jywarren commented 3 years ago

We spend a lot of compute time encoding images as datauri in order to pass them from one module to the next. It may also be possible to use blobs and save encoding time, although we would definitely want to compare the benchmark tests to see if this is true.

(moving this from this comment thread)

I think this is probably worth it! I think it's possible. We had very very long ago considered different formats for between-module data handoff, including possibly buffers...? i forget, but it was in our very first issue: https://github.com/publiclab/image-sequencer/pull/1#issuecomment-288275324

the default is data-urls, because strings are completely compatible, but if two modules can negotiate that both are compatible with blobs, that's OK too. I think originally the data-urls worked especially well because they had to be displayed anyways. But now that I think of it I think it's possible to display a blob as well in the browser.

Also note https://stackoverflow.com/questions/7650587/using-javascript-to-display-a-blob

I wonder if this would just work, but it depends on the run environment, I imagine:

https://github.com/publiclab/image-sequencer/blob/5298d1e0ecadb83a02111342a72dc3690b27a6ad/src/Run.js#L18-L54

We could add a parameter to drawStep somewhere that evaluates whether the neighboring steps can both handle blobs.

Interestingly i think we may already be doing this in some modules, where interacting with the glfx library:

https://github.com/publiclab/image-sequencer/blob/3caa033c8febf5055c28f616b534933c00fcbe65/src/modules/ColorHalftone/Module.js#L35-L36

See this section:

https://github.com/publiclab/image-sequencer/blob/3caa033c8febf5055c28f616b534933c00fcbe65/src/modules/ColorHalftone/Module.js#L71-L82

there, we are trying to treat a blob URL just like a url by using blobUrl = window.URL.createObjectURL(blob); !!

jywarren commented 3 years ago

cc @publiclab/is-maintainers

jywarren commented 3 years ago

And noting that this would also require documentation changes to the Module API to make devs aware that this is allowed.