Enable Asynchronous Processing

[harshkhandeparkar]

Asynchronous

The Image Sequencer processes can be run on an alternate thread using webworkers. Native support for workers has also been added to Node.js. Greenlet(workerize) can also be used.

Use Case

Doing this will not slow down the browser as much as it did before. All other browser activities will work and sequencer will run on an alternate thread and only a spinner will be shown.

Ideas

Do you have any other ideas? Any other ways to tackle this? Anything? Please let us know in the discussion(comments).

---

Thank you!

Your help makes Public Lab better! We deeply appreciate your helping refine and improve this site.

To learn how to write really great issues, which increases the chances they'll be resolved, see:

https://publiclab.org/wiki/developers#Contributing+for+non-coders

[jywarren]

Hi Harsh! Could you link back to the original issue and copy in some of the links to libraries and docs that could help? I think there was a library called Greenlet or something. Thanks!

On Sun, Mar 3, 2019, 1:49 PM Harsh Khandeparkar notifications@github.com wrote:

Asynchronous

The Image Sequencer processes can be run on an alternate thread using webworkers. Native support for workers has also been added to Node.js Use Case

Doing this will not slow down the browser as much as it did before. All other browser activities will work and sequencer will run on an alternate thread and only a spinner will be shown. Ideas

Do you have any other ideas? Any other ways to tackle this? Anything? Please let us know in the discussion(comments).

Thank you!

Your help makes Public Lab better! We deeply appreciate your helping refine and improve this site.

To learn how to write really great issues, which increases the chances they'll be resolved, see:

https://publiclab.org/wiki/developers#Contributing+for+non-coders

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/829, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ8_g0MpwPVo8rhHxlWC-t3nQuhaoks5vTBkHgaJpZM4bbKYN .

[jywarren]

Oh sorry I read from my email, didn't see it was done!

[jywarren]

And https://github.com/publiclab/image-sequencer/issues/159

[Divy123]

I am new to web workers but I would love to work upon this. @HarshKhandeparkar @jywarren waant your guidance here. Please help me with this like what part needs to run on alternate thread and following from the discussion in #159, for the browser, what can be a good resource to implement it with? Since this is a performance issue, I want to work on it at a top priority. Thanks!!

[harshkhandeparkar]

Since nodejs has workers support, can you first try running different parts of sequencer on different threads? You will have to do some research on workers though.

[Divy123]

For sure! Thanks a lot. Also want to know your opinions @jywarren

[Divy123]

@jywarren following #159, do you suggest to have run() on another thread ? Also what are some other parameters you suggest keeping in mind? Thanks!! PS: Know that you are very busy but actually I am drafting my soc proposal for that I am exploring web workers and related concepts.

[Divy123]

Greenlet is most effective when the work being done has relatively small inputs/outputs.

Also I found this on greenlet docs. Will that be fruitful using @jywarren ?

[jywarren]

I think we basically need to try it and see, unfortunately. I guess this is a vote in favor of doing the whole run() rather than passing back and forth to/from the worker thread... what do you think?

On Fri, Mar 29, 2019 at 6:21 PM Slytherin notifications@github.com wrote:

Greenlet is most effective when the work being done has relatively small inputs/outputs. Also I found this on greenlet docs. Will that be fruitful using @jywarren https://github.com/jywarren ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/829#issuecomment-478166974, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ-yRuUfC8bFjLFh7_BFwoqMsTxL2ks5vbpHmgaJpZM4bbKYN .

[Divy123]

Yes I think that has to be the soution in that case. But one thing to keep in consideration is we cannot manipulate DOM from worker thread so some utility has to be provided to worker or returned by it to do the task outside worker thread.

[jywarren]

ah, indeed. So is it possible to use some kind of callback with the image from each step?

On Fri, Mar 29, 2019, 7:34 PM Slytherin notifications@github.com wrote:

Yes I think that has to be the soution in that case. Yes but one thing to keep in consideration is we cannot manipulate DOM from worker thread so some utility has to be provided to worker or returned by it to do the task outside worker thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/829#issuecomment-478179722, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ3YBsYsz0ySp0KMQMI3NmM1HJAZpks5vbqLRgaJpZM4bbKYN .

[Divy123]

Will get to you back after exploring a bit more on this as I think the problem is since the data is copied not shared, returning images(heavy data) has to be optimized.

[Divy123]

Since greenlet does not work with nodejs, I went through node-webworker. But the issue I am facing is:

internal/bootstrap/loaders.js:81
        mod = bindingObj[module] = getBinding(module);
                                   ^

Error: No such module: net
at process.binding (internal/bootstrap/loaders.js:81:36)
    at Object.<anonymous> (/home/divy/Desktop/practice/node_modules/webworker/lib/webworker.js:35:26)
    at Module._compile (internal/modules/cjs/loader.js:701:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
    at Module.load (internal/modules/cjs/loader.js:600:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
    at Function.Module._load (internal/modules/cjs/loader.js:531:3)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.<anonymous> (/home/divy/Desktop/practice/main.js:1:76)

So I think we can make this happen only on browsers until we don't find a good solution for node. @jywarren what do you think upon this?

[Divy123]

Also I am exploring the following for node: https://www.npmjs.com/package/webworker-threads

[harshkhandeparkar]

@Divy123 are you using a separate package? Doesn't nodejs have inbuilt support for workers? I think it should have because some node libs use promises, don't they? Also I think Varun had said that node has support for workers in a comment in the old issue.

Yes, he did here https://github.com/publiclab/image-sequencer/issues/159#issuecomment-395843135

cc @jywarren @tech4GT

[harshkhandeparkar]

News!

Nodejs v11.7.0 has worker threads inbuilt.

https://codeforgeek.com/2019/01/getting-started-node-worker-thread/

[harshkhandeparkar]

v10.5.0 has initial implementation of threads and v11.7.0 has full worker threads support.

[Divy123]

@HarshKhandeparkar latest version of node is 10.15.0 So we shoud proceed that way. We can use the native support in node v11.7.0 once it becomes the LTS but till then, I think we can proceed that way.

[Divy123]

@jywarren request a little help here. Can you please help me out with how can I pass functions to workers since they are not allowed as per structured clone algorithm which web workers employ to copy data?

[harshkhandeparkar]

So we shoud proceed that way. We can use the native support in node v11.7.0 once it becomes the LTS but till then, I think we can proceed that way.

This version won't be an LTS, node v12 will be. But anyways, :+1:

[harshkhandeparkar]

request a little help here. Can you please help me out with how can I pass functions to workers since they are not allowed as per structured clone algorithm which web workers employ to copy data?

Can you drop a link to the docs/README of the package you are using?

[Divy123]

@HarshKhandeparkar may have a look https://www.npmjs.com/package/webworker-threads

[harshkhandeparkar]

Can you please explain a bit more? What is structured clone algorithm?

[harshkhandeparkar]

What function do you want to pass into the worker?

[Divy123]

Structured clone algo is the one used to copy data from ine worker to another and it has certain restrictions. Can have a look here: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm

[Divy123]

Also I have to pass certain number of functions including the one that are imported through require() as require cannot be used inside worker threads. So there is a lot to pass. For example, I want to have run() on a worker thread and for that I have to certain functions like callback() to it. Do you have any suggestions @HarshKhandeparkar @jywarren ?

[harshkhandeparkar]

1. MDN docs document browser workers, node may be different.
2. You can create workers inside the run func, pass static data through postMessage and let the worker return the final pixels, again using postMessage.
3. You don't have to use require inside the workers, you can require outside and provide it to the worker.
4. This is from the docs,

   .importScripts( file [, file...] ) importScripts('a.js', 'b.js') loads one or more files from the disk and eval() them in the worker's instance scope.

Whatever I have said may not work. You will have to experiment. Correct me if I have said anything wrong. Please feel free to ask other queries(if any).

[harshkhandeparkar]

Try this,

var Worker = require('webworker-threads').Worker;
// var w = new Worker('worker.js'); // Standard API

// You may also pass in a function:
var worker = new Worker(function(){
  postMessage("I'm working before postMessage(function(var){return 2*var}).");
  this.onmessage = function(event) {
    postMessage(event.data(4)); // 8 (should be)
    self.close();
  };
});
worker.onmessage = function(event) {
  console.log("Worker said : " + event.data);
};
worker.postMessage(function(var){return 2*var});

[Divy123]

Actually theimportScripts() runs the script most probably inside the file provided to it so the functions provided inside it will not work and moreover we cannot pass any args to it. I will think about restructuring somethings here. Thanks a lot @HarshKhandeparkar

[Divy123]

Looking at the code ..

[Divy123]

Also @HarshKhandeparkar can you please provide a link to the above code?

[Divy123]

Also the code provided is not running, it get stuck after logging

worker said : I'm working before postMessage(function(var){return 2*var})

to the console.

[Divy123]

@jywarren request for your help here!!

[Divy123]

Also @tech4GT can you please help me with this. I am including this in my SoC proposal, so I don't have much time left.

[Divy123]

I think we can break various parts of run function and have them on alternate threads. What do you say on this? Like inside Run.js file we can run filter() on a thread and then utilize its results inside drawSteps() where again we can use a new thread to perform all the calculations above drawStep() funcion call. And similarly we can break different parts of the remainig file like this only.

[harshkhandeparkar]

Also the code provided is not running, it get stuck after logging

to the console.

Thia should have logged Worker Said : 8

Ok, let's try something simpler.

var Worker = require('webworker-threads').Worker;
// var w = new Worker('worker.js'); // Standard API

// You may also pass in a function:
var worker = new Worker(function(){
  postMessage("I'm working before postMessage(function(var){return 2*var}).");
  this.onmessage = function(event) {
    postMessage(event.data()); // 2 (should be)
    self.close();
  };
});
worker.onmessage = function(event) {
  console.log("Worker said : " + event.data); // Worker Said : 2
};
worker.postMessage(function(var){return 2});

I didn't find this code anywhere, I just edited the example code.

[harshkhandeparkar]

You can try something even simpler, (I'm just trying to pass a function into the worker using postMessage).

var Worker = require('webworker-threads').Worker;
// var w = new Worker('worker.js'); // Standard API

// You may also pass in a function:
var worker = new Worker(function(){
  postMessage("I'm working before postMessage");
  this.onmessage = function(event) {
    postMessage(typeof event.data); // function (should be)
    self.close();
  };
});
worker.onmessage = function(event) {
  console.log("Worker said : " + event.data); // Worker Said : function
};
worker.postMessage(function(var){console.log('hi')});

[Divy123]

@HarshKhandeparkar this is something that supports both browser and node . https://www.npmjs.com/package/threads

[Divy123]

I think this is one of the most reliable library, has 11032 weakly downloads .

[niravasher]

This is what I have understood till now regarding web workers

This is the way I am applying web workers to the DrawRectangle Module workers.onMessage() is outside extraManipulation() and inside workers.postMessage, I am returning output, changePixel, extraManippulation, format, image and callback. This might not be entirely right but I am having a fair idea about workers, require some help here. @jywarren @HarshKhandeparkar @Divy123

[Divy123]

@niravasher I am already working upon it.

[Divy123]

Also we first focus on Run.js file which is so much used.

[jywarren]

Hi all, just linking to #1069 and adding resources -

• http://madhugnadig.com/articles/parallel-processing/2017/03/29/the-guide-to-parallel-programming-in-javascript.html
• https://parallel.js.org/