Investigate creating a worker thread

[1fifoto]

Using Web Workers

[1fifoto]

Use the following Article to investigate web worker modules : https://web.dev/module-workers/

[1fifoto]

The initial code in branch 787 is more a version of the first article above and not the second article.

[grumpyinca]

Regarding the transfer of data back to the main thread from the worker thread ...

It is my understanding that there needs to be an explicit extraction of information from e.data, the worker thread's returned message.

Per discussion, there are two choices in updating the model with the values returned by the worker thread:

1. Over-write all or most of the symbol table and objective value with the returned values

2. Extract & over-write only the returned free independent variables followed by an execution of eqnset.

   • Alternative one is likely more general and extensible to situations that go beyond search. Also, it is computationally efficient and thus more scalable to models with great computational requirements.

   • Alternative two is better aligned with the current code in search.js. I believe the values are extracted in lines 33 to 42 and the model (design) updated on line 47. This approach is also better aligned with the way that I think about the problem and perhaps a bit more robust. Extracting only the values that are associated with free independent variables into the model (symbol table) and then executing eqnset to get the corresponding dependent variables and objective value incurs the least amount of risk of unexpected changes being introduced into the model. Considering that a search typically involves hundreds or even many thousands of evaluations of eqnset, I don't think that there is a practical difference in performance in the two alternatives as observed by the user.

Having said all this, I will add that I think that both alternatives get the job done. I do not have a strong preference.

[1fifoto]

import PropTypes from 'prop-types';

  const { store } = this.context;
  var design = JSON.parse(JSON.stringify(store.getState())); // local non-React clone
  const local_store = createStore(reducers, design, applyMiddleware(dispatcher));
//      console.log('store=',store,'design=',design,'local_store=',local_store);
  var local_design = local_store.getState();

  local_store.dispatch(search());

PlotValue.contextTypes = {
    store: PropTypes.object
};

[1fifoto]

We read: React + Redux + Comlink = Off-main-thread. The author is pushing OMT Off-main-thread architecture. It was a good article and suggested using Comlink and putting our entire Redux code into the worker thread, which is quite a shift in thinking for us. He used Rollup to package the worker and all it's code into a loadable chunk. He provides a sample as a gist. This article referenced a blog post Is postMessage slow? explaing that updating the main thread state in bulk can be reduces to smaller patches using ImmerJS. Possible YouTube video The main thread is overworked & underpaid (Chrome Dev Summit 2019)

[1fifoto]

Continuing investigation...

We have two possible approached to pursue. The first is to try to use CRA in ODOP to incorporate web workers with is done by webpack and possibly babel. The second is to migrate ODOP to use Rollup as the module bundler. It is felt that is is the CRA approach is the better one.

Following the CRA path... We have found out today that webpack v4 provided 'worker-loader' to support webworkers, but it has since been deprecated and replaced in webpack v5 with the following. See this for more information.

// or customize the chunk name with magic comments
// see https://webpack.js.org/api/module-methods/#magic-comments
new Worker(
  /* webpackChunkName: "foo-worker" */ new URL('./worker.js', import.meta.url)
);

So we have decided to integrate the sample app that was used by the Rollup approach and migrate into the CRA approach to test the CRA Webpack work.

[1fifoto]

We got the CRA approach working with the sample app from the Webpack documentation. Attached is the zip file of the Eclipse project webworkerWithCRA webworkerWithCRA.zip

[1fifoto]

Began working on integrating redux and comlink into our working webworkerWithCRA as a new workspace named webworkerWithCRAComlink. The application is nearly working, but presently get a "Unserializable return value" when it tries to do a postMessage passing in an unsubscribe function which cannot be serialized. Why this is happening is still unknown.

Continuing... In "https://github.com/GoogleChromeLabs/comlink" closed issue 341 which is titled "React-redux store example: proxy susbcribe callback fails". See Details by surma who commented on Aug 16, 2019.

[1fifoto]

See slorber/react-async-hook for displaying a spinner, waiting until the dispatch completes, and getting rid of the spinner.

[1fifoto]

After considerable work, the remote-store-wrapper.js function was missing 'return' on its output parameter, because Brian converted to the function from a thick-arrow representation.

From:

dispatch: action => store.dispatch(action),

to this faulty conversion

dispatch: (action) => {
  console.log('dispatch action=',action);
  store.dispatch(action)
},

whereas it should have been

dispatch: (action) => {
  console.log('dispatch action=',action);
  return store.dispatch(action) // Notice the RETURN is now present
},

After this simple but very important repair, the sample in (1) webworkerWithCRAComlinkSpinner and (2) webworkerWithCRAComlinkODOP now are working. That is, the spinner appears when (1) executing the delay function or (2) computing the Search and disappears when (1) the delay function completes or (2) the Search completes.

Attached is the webworkerWithCRAComlinkSpinner and webworkerWithCRAComlinkODOP zip file with the latest code as of today.

webworkerWithCRAComlinkSpinner20240211.zip

webworkerWithCRAComlinkODOP20240211.zip

[1fifoto]

We next attempted to merge these changes from the working samples (see comment above) into the 787 branch and ran into problems. The code got the following error

Failed to compile

./src/index.js 39:97
Module parse failed: Unexpected token (39:97)
You may need an appropriate loader to handle this file type.
| //console.log('<li>','Start index.js','</li><ul>');
| 
> var worker = new Worker( /* webpackChunkName: "odop-worker" */new URL('./store/worker.js', import.meta.url));
| console.log('in index.js worker=', worker);
| var remoteStore = wrap(worker);

This error occurred during the build time and cannot be dismissed.

We attempted to make changes, for example removing "import.meta.url" from the Worker statement above, but we were unable to resolved them. This cascaded into flagging "await". It is our belief that it is due to the sample being at the latest version of React (verseion 18) and the 787 branch being at an older version (versiuon 16). But this is just a supposition.