Add WebWorker Support

[dansiviter]

Add ability to load WebWorkers (and SharedWorkers?) via the react-scripts. My use-case is for using latency (and debug) sensitive protocols over WebSocket which may get dumped by the server if no activity occurs.

There has been a lot of discussion regarding this on #1277 but no definitive answer and now closed. Also, I'm a server-side dev so JavaScript PR is not my forte, and neither is ejecting the scripts as that looks scary.

[gaearon]

I’m not opposed if we see a clear proposal from someone about how they should work.

[viankakrisna]

I haven't written WebWorkers before, but I think this library is interesting to get support of web workers in CRA right now: https://github.com/developit/workerize

it also has it's own webpack loader https://github.com/developit/workerize-loader (not required to use workerize)

which in turn inspired by https://github.com/webpack-contrib/worker-loader

I think adding the last one with test like .worker.jswould be a good option if we want to support it out of the box without tying it with any library (except webpack?)

[iansu]

workerize seems like a more straight forward solution. It looks like it handles the communication between the parent thread and the worker, which is nice. You just have to export a method from the worker as opposed to using postMessage/onmessage to communicate between the two.

I like the idea of using a different extension (.worker.js) to signify a worker and having it registered automatically. That seems to fit nicely with create-react-app.

[iansu]

I'm curious about WebWorkers in general so I wouldn't mind giving this a try.

[wregis]

In the past I had to eject and configure "worker-loader". I used it to process some routines on datasets and still allow users to interact with the main UI.

[doxxx]

I use Web Workers to implement a solver using genetic algorithms for a SPA. The solver can run for up to a minute or two, so it needs to be in a "background thread" -- i.e. Web Worker -- to not freeze the browser page while it's running.

I should note that my app is currently implemented in Angular and plain old ES5-ish Javascript. I would like to migrate to React and Typescript but this has been a stumbling block so far.

[jlarmstrongiv]

Is there anything that needs to be done before this pull request can be merged into the next branch (2.0 release)? I might have missed it, but I could not find Web Workers on the 2.0 Projects roadmap. I would love to be able to try it out in one of the alpha releases 🙂

[iansu]

I think there's still some debate about using worker-loader vs. workerize-loader. worker-loader is more low-level and powerful but workerize-loader is incredibly easy to use and probably covers the majority of use-cases.

[doxxx]

I couldn't get workerize-loader to work with Typescript, but I may have simply failed at the appropriate incantations.

[Narvey]

@iansu I'm not sure about workerize-loader covering the majority of use-cases. Unless I'm understanding incorrectly, it doesn't allow sending messages in the middle of a loop (this is extremely important for making realistic progress bars).

[npotts]

I realize there is a desire to make a 'simpler' API for web workers, but honestly, the basic subset is more than sufficient. I have a data intensive app, and want to use web workers to do the data fetches, parsing, and pass massaged data back to the main thread often.

A 'run once - read once' methodology may work for some applications, but nothing that I am involved with.

[doxxx]

Ah, yeah, workerize will not work (heh) for me. I need to keep a web worker running and exchange messages back and forth. Progress bars is one of the uses as @Narvey mentioned.

[gpolyn]

Granted a distinction between "physical" threads and "software" threads, such that creating many web workers may result in more software threads than available physical threads, a developer may wish to mind the user's hardware utilization and create fewer or no more than the available number of physical threads, e.g., https://github.com/josdejong/workerpool/blob/master/lib/Pool.js.

So, here are two React pooling attempts:

• worker-loader: https://github.com/gpolyn/react-and-worker-loader
• workerize-loader: https://github.com/gpolyn/react-and-workerize-loader

[sinedied]

Would love to also see some support for newly introduced AudioWorklet, which are basically web worker for audio processing 😃

[jasmith79]

Another use case:

Desktop Chrome (~75% global market share) throttles setInterval in unfocused tabs. If you want to ensure that something is running the easiest way is to run it in a Worker.

[doxxx]

@jasmith79 To be honest, that sounds like a hack to get around a legitimate browser restriction. What's the actual use case?

[jasmith79]

@doxxx I wrote an app for work that has to be able to do background update, so I poll the backend for changes every so often on an interval timer. Got reports that it was wonky in Chrome (worked fine in FF/Safari) and discovered that (this was before serviceworker). Runs fine in a webworker. Now I'm porting the front end to react and discovered this problem.

[gampleman]

I'm wondering if we could write something like workerize, but with the following changes:

• exporting a function from a worker module that is not async will cause a build time error (as opposed to workerize "silently" turning it async for you)
• Supports async generator functions. That is if a function is an async generator than it can communicate intermediate results or progress to the main thread. The main thread can even pass information in via the next method.

I think these changes would achieve the following goals:

1. For simple use cases we keep the simplicity and elegance of workerize.
2. I think this should be nearly as expressive as the low-level api of web workers, but nicer to work with.
3. The module would work exactly the same if the worker loader would disappear, except it would run on the same thread (but async). This would make it relatively low commitment and also easy to test, since the test framework doesn't need to worry about worker support (which can be slightly tricky in node).

How does that sound?

[iansu]

That does sound useful but it also sounds like it's outside the scope of Create React App. Have you considered proposing these changes to Workerize?

[gampleman]

I was more thinking of building this as a separate library. But I'm quite interested if that proposed design would fit the needs that people have for webworkers and if something like that would fit into cra?

[doxxx]

All I really want is just for the existing WebWorker APIs to work and to be able to load a js/ts file as a web worker. Anything beyond that lies outside the scope of CRA as far as I’m concerned. That forms the basis on which everything else can be built.

[breznik]

I know most of the conversation is around implementing support for Web Workers, but Web Worker usage is the gateway to then needing to use SharedWorkers, so it would be really amazing if there was a way to handle the ad-hoc solutions needed today if you want to utilize Web Workers, SharedWorkers, or AudioWorklets - or in my case, the combination of all three. Conventions for special treatment such as .worker.js, .sharedworker.js, .worklet.js (or .awn.js?) would work wonders for those of us using CRA for more advanced cases.

[stramel]

https://github.com/GoogleChromeLabs/worker-plugin

[iansu]

I saw that the other day. It looks very interesting! I’m planning to look into it in more detail and see if it fits our usecase.

[gaearon]

Yeah I think we should probably add it

[iansu]

Sounds good. I’ll update this PR.

[gampleman]

Does that plugin not break referential transparency in a rather bad way?

[npotts]

@gampleman

Does that plugin not break referential transparency in a rather bad way?

Could you give an example that demonstrates this?

[arizonatribe]

Kind of ironic, but it's written by the same author who created workerize and workerize-loader, which presented itself when referencing self in the worker I tested it out with:

Warning (workerize-loader): output.globalObject is set to "window". It should be set to "self" or "this" to support HMR in Workers.

I ran into difficulty yesterday adding it and testing it out in our company's ejected CRA project. We originally tried to go the route of workerize but ran into an issue getting it to work at all in standard Chrome, and it was a bit concerning for us that the author showed there - and on several other GitHub issues I've observed for his tools - he may never respond at all to a question. In our case we demonstrated errors using workerize identically to his example, but the only response we ever got was it only works in Chrome Canary (docs never updated though).

But for this plugin (worker-plugin) I'm currently not able to complete a build with a worker instantiated as {type: 'module'}, identical to his example.

It throws this error about Dedicated Worker not being supported yet and kicks it over the wall to a Chromium bug thread that doesn't look like it's going anywhere for awhile

Uncaught TypeError: Failed to construct 'Worker': Module scripts are not supported on DedicatedWorker yet. You can try the feature with '--enable-experimental-web-platform-features' flag (see https://crbug.com/680046)

I'm not sure (in our case) we can convince the users that come to our site to use Canary or to go to their Chrome flags and turn on that one he mentioned.

[gampleman]

I mean that

const worker = new Worker('./foo.js', { type: 'module' });

will work, but

const workerPath = './foo.js';
const worker = new Worker( workerPath, { type: 'module' });

will not. Neither will

const options =  { type: 'module' };
const worker = new Worker('./foo.js', options);

[NicolasRannou]

Any update on that?

I saw that the related PRs have been closed without comments, so I'm wondering if I should move away from create-react-app if I want to use web workers or if there is still a plan to support it!

Thanks

[0xcaff]

From my understanding, it seems that no consensus has been on which solution will be chosen.

Options

worker-plugin

Not great for the reasons mentioned by @gampleman. Namely, worker-plugin transforms the Worker constructor at build time leading to some strange behavior. No worklet support (https://github.com/GoogleChromeLabs/worker-plugin/issues/7)

workerize-loader

An abstraction to push work to web workers. Maybe too high level. Maybe has bugs. Doesn't support worklets. Probably not the best way to do worklets.

worker-loader

A low level abstraction for web workers. This seems to be the recommended way to bundle workers with webpack. Doesn't support worklets.

I think worker-loader is the best choice. None of these support worklets. Worklets should probably be figured out later so we don't get locked into a bad choice.

[iansu]

I agree that worker-loader is probably the best solution right now. I've created an updated version of my previous PR that adds worker-loader: https://github.com/facebook/create-react-app/pull/5886

[NicolasRannou]

OK, thanks!

The sad part is that it all requires create-react-app to be ejected which voids the warranty :)

Thanks anyway!!

[iansu]

The point of adding this to Create React App is that you won't have to eject.

[NicolasRannou]

Yes exactly, using your PR we do not need to eject.

Hopefully, that will be integrated at some points since those are now an important part of the web ecosystem!

[arizonatribe]

@NicolasRannou Until a solution is created, have you tried using react-app-rewired to allow you to add in worker-loader? I had luck with that tool when I wanted to use workbox earlier this year (before CRA switched over to use it). I'm sure it'll still void your warranty, but it won't be as messy to drop it as it would to un-eject later.

[A]

Hey guys! I see all 3 solutions you've mentioned handle only WebWorker, but not SharedWorker, that's not so sweet. One of greatest usecases of SharedWorker is sharing websocket connection between multiple tabs, that reduces network traffic dramatically https://github.com/pusher-community/pusher-with-shared-workers. Now it's the only one reason we're still ejected. @viankakrisna @gaearon @iansu can I help you guys to implement and release it?

[jonathanzong]

Hey all, I'd be excited to see this land in create-react-app.

One potential problem I encountered was that when running npm run build, the build output is empty if and only if I am importing a worker in my project code. More specifically, when the build script completes, my build/ folder will only contain the contents copied from public/ and nothing else.

This happens both when I use react-app-rewire AND when I clone iansu:worker-loader and link react-scripts locally.

Has anyone else encountered this?

[pavelserbajlo]

same here. Build passes without errors, but only public files are copied. Removing the import sorts the build out. Any way to get some verbose build logs to find the culprit here?

[MehmetKaplan]

Is there any working solution (whether or not a workaround) on this issue?

I tried scenarios that I foıund online but none worked properly. (I don't want to eject CRA.)

So any suggestion or guidence are appreciated...

Edit: My case is solved as https://stackoverflow.com/a/55433737/1870873 (Just in case it may help someone).

[petersobolev]

For my project I'm using solution with blob. Here is example: https://github.com/petersobolev/cra-worker

[usworked]

Are webworkers supported in the latest v3?

[MatteoGioioso]

Any update on this?

[mib32]

For anyone using the react-app-rewired, just paste this into config-overrides.js (mostly stolen from @iansu)

const fs = require('fs');
const path = require('path');
const appDirectory = fs.realpathSync(process.cwd());
const resolveApp = relativePath => path.resolve(appDirectory, relativePath);
const getCacheIdentifier = require('react-dev-utils/getCacheIdentifier');

const isEnvProduction = true // TODO: insert your implementation
const isEnvDevelopment = false // TODO: insertyour implementation

module.exports = function override(config, env) {
  config.module.rules[2].oneOf.splice(1, 0, {
      test: /\.worker\.js$/,
      include: resolveApp('src'),
      use: [{ loader: require.resolve('worker-loader'), options: {
        inline: false
      } },
        {
          loader: require.resolve('babel-loader'),
          options: {
            customize: (
              require.resolve('babel-preset-react-app/webpack-overrides')
            ),
            babelrc: false,
            configFile: false,
            presets: [require.resolve('babel-preset-react-app')],
            // Make sure we have a unique cache identifier, erring on the
            // side of caution.
            // We remove this when the user ejects because the default
            // is sane and uses Babel options. Instead of options, we use
            // the react-scripts and babel-preset-react-app versions.
            cacheIdentifier: getCacheIdentifier(
              isEnvProduction
                ? 'production'
                : isEnvDevelopment && 'development',
              [
                'babel-plugin-named-asset-import',
                'babel-preset-react-app',
                'react-dev-utils',
                'react-scripts',
              ]
            ),
            plugins: [
              [
                require.resolve('babel-plugin-named-asset-import'),
                {
                  loaderMap: {
                    svg: {
                      ReactComponent:
                        '@svgr/webpack?-prettier,-svgo![path]',
                    },
                  },
                },
              ],
            ],
            // This is a feature of `babel-loader` for webpack (not Babel itself).
            // It enables caching results in ./node_modules/.cache/babel-loader/
            // directory for faster rebuilds.
            cacheDirectory: true,
            cacheCompression: isEnvProduction,
            compact: false
          },
        }
      ]
    })
  config.output['globalObject'] = 'self';

  return config;
}

[antoniotorres]

Update?

[wangkailang]

Any good resolutions?

[viankakrisna]

If anyone felt blocked to use a Web Worker with CRA right now, you could already do that with placing the worker file in the public folder and referencing it on the Worker constructor as described here https://github.com/facebook/create-react-app/issues/1277#issuecomment-267322485

ie:

// public/counterWorker.js
var i = 0;

function timedCount() {
  i = i + 1;
  postMessage(i);
  setTimeout("timedCount()",500);
}

timedCount();

// src/index.js
const counterWorker = new Worker("/counterWorker.js");
counterWorker.onmessage = (event) => {
   console.log(event.data)
}

The worker file won't be transpiled and minified when deploying, you'll need to handle that yourselves.

[Krivega]

small clarification

// public/worker.js

// some worker

// src/index.js
const getPathFromPublic = path => `${process.env.PUBLIC_URL}/${path}`;

const workerPath = getPathFromPublic('worker.js');
const worker = new Worker(workerPath);

[NicolasRannou]

FYI, I use 'worker-loader' + 'comlink' to manage my web-workers and it seems to work fine.

The trick is to use the inline syntax to use the 'worker-loader' so we do not have to edit the webpack configuration.

main.ts file

/* eslint-disable import/no-webpack-loader-syntax */
import Worker from 'worker-loader!../util/worker/Worker';
import * as Comlink from 'comlink';
import { WorkerType } from '../util/worker/Welcome';

...

 async function startDummyWorker() {
    const workesr = new Worker();
    const obj = Comlink.wrap<WorkerType>(workesr);
    const finished = await obj.busy(10000000000000000000);
    window.console.log(finished);
  }
  startDummyWorker();
...

worker.ts file

export interface WorkerType {
  busy(value: number): number;
}

const obj: WorkerType = {
  busy(value: number) {
    let j = 0;
    const start = Date.now();

    for (let i = 0; i < value; i++) {
      //
      j++;
      const end = Date.now();
      // if it has been busy for more than 10s return
      if (end - start > 10000) {
        return 42;
      }
    }
    return j;
  },
};

Comlink.expose(obj);