GoogleChromeLabs / comlink-loader

Webpack loader to offload modules to Worker threads seamlessly using Comlink.
https://npm.im/comlink-loader
Apache License 2.0
620 stars 40 forks source link

support for webpack 5 #34

Open danutzcodrescu opened 3 years ago

danutzcodrescu commented 3 years ago

It seems that webpack 5 in production mode breaks the singleton mode (that was working fine for webpack 4). It throws an error that __webpack_exports__ is not defined . In development mode for webpack 5 everything works as expected, but only in production it generates the error. I identified that if I set in webpack optimization usedExports: false then the issue is resolved (however that is an undesirable option).

This is the config for webpack:

module.exports = merge(baseConfig, {
  target: 'electron-renderer',
  entry: {
    app: './src/client/app.tsx',
  },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        exclude: /node_modules/,
        use: {
          loader: 'babel-loader',
          options: {
            cacheDirectory: true,
            babelrc: true,
          },
        },
      },
      {
        test: /\.css$/,
        use: ['style-loader', 'css-loader'],
      },
      {
        test: /\.(gif|png|jpe?g)$/,
        use: [
          'file-loader',
          {
            loader: 'image-webpack-loader',
            options: {
              bypassOnDebug: true,
            },
          },
        ],
      },
      {
        test: /\.svg$/,
        use: ['svg-inline-loader?classPrefix'],
      },
      {
        test: /\.md$/i,
        use: ['raw-loader'],
      },
      // All output '.js' files will have any sourcemaps re-processed by 'source-map-loader'.
      {
        enforce: 'pre',
        test: /\.js$/,
        use: ['source-map-loader'],
      },
      {
        test: /\.worker\.ts$/i,
        use: [
          {
            loader: 'comlink-loader',
            options: {
              singleton: true,
            },
          },
          {
            loader: 'babel-loader',
            options: {
              cacheDirectory: true,
              babelrc: true,
            },
          },
        ],
      },
    ],
  },
  plugins: [
    new ForkTsCheckerWebpackPlugin({
      typescript: {
        diagnosticOptions: {
          semantic: true,
          syntactic: true,
        },
      },
    }),
    new HtmlWebpackPlugin({ template: path.resolve(__dirname, 'src/client/index.html') }),
    new webpack.DefinePlugin({
      'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV || 'development'),
      'process.env.STORE': JSON.stringify(process.env.STORE || ''),
    }),
    new CopyPlugin({
      patterns: [{ from: path.resolve(__dirname, 'static/'), to: path.resolve(__dirname, 'dist', 'static') }],
    }),
  ],
});

This is the babel config:

module.exports = (api) => {
  // This caches the Babel config by environment.
  api.cache.using(() => process.env.NODE_ENV);
  const developmentPlugins = ['@babel/plugin-transform-runtime', 'react-refresh/babel'];

  const productionPlugins = [
    // babel-preset-react-optimize
    '@babel/plugin-transform-react-constant-elements',
    '@babel/plugin-transform-react-inline-elements',
    'babel-plugin-transform-react-remove-prop-types',
  ];
  const development = api.env('development', 'test');

  return {
    presets: [
      ['@babel/preset-env', { targets: 'last 1 chrome version' }],
      '@babel/preset-typescript',
      '@babel/preset-react',
      '@emotion/babel-preset-css-prop',
    ],
    plugins: [
      ['@babel/plugin-proposal-decorators', { legacy: true }],
      ['@babel/plugin-proposal-class-properties', { loose: true }],
      '@babel/plugin-proposal-optional-chaining',
      '@babel/plugin-proposal-nullish-coalescing-operator',
      ...(development ? developmentPlugins : productionPlugins),
    ],
  };
};

It works fine in webpack 4 (dev and prod) and only in dev mode in webpack 5. Any ideas how can we improve comlink support for webpack 5 in prod?

lionelhorn commented 3 years ago

I'm getting the same __webpack_exports__ is not defined error in prod. Webpack 5. No error in development mode.

geakstr commented 3 years ago

Same for me, works in dev, broken in prod. Workaround with webpack optimization.usedExports: false suggested by @danutzcodrescu helps though.

lianghx-319 commented 3 years ago

I got an error in only webpack 5 prod like that. image Anyone has same issue ?


optimization.usedExport: false it seem a workaround for me

kamikat commented 2 years ago

@lianghx-319 Got Cannot read property 'apply' of undefined in webpack 5 production build.

But in my case, turning off tree shaking throughout the entire project by optimization.usedExport: false should not be an ideal solution.

And here's another workaround for singleton mode:

// in worker module
export const doSomething1 = ...;
export const doSomething2 = ...;

// @ts-ignore
__webpack_exports__ = { doSomething1, doSomething2 };
VittorioAccomazzi commented 2 years ago

I ran in to this problem as well using the new Create React App, which now includes WebPack 5.0. The problem is that WebPack 5.0 doesn't uses worker-loader since it supports web worker natively, details here.

The way I solved this problem was simply to use the functionalities in WebPack 5.0 and then wrap the code with ComLink as follow:

PrimeNumber.ts (I add a parameter on the constructor on purpose for testing):

import {expose} from 'comlink';
export type PrimeNumberClassConstructors = { new ( num : number ): PrimeNumber };

export default class PrimeNumber {
    private num : number =0;

    constructor ( num : number  ){
        this.num = num;
    }

    public generate( ) {
        let list : number [] = [2];
        let val = 3;
        while( list.length < this.num ){
            let isPrime = list.every((v)=>(val%v)!==0);
            if( isPrime ) list.push(val)
            val ++;
        }
        return list;
    }
}

expose(PrimeNumber)

and the I use the worker as follow:

import * as Comlink from 'comlink'
import PrimeNumber, {PrimeNumberClassConstructors} from './PrimeNumber';

export default class PrimeNumberProxy {
    private worker : Worker;
    private proxy : Comlink.Remote<PrimeNumber>|null;
    private num : number = 0;

    constructor ( num : number ){
        this.worker = new Worker( new URL('./PrimeNumber.ts',import.meta.url));
        this.proxy = null;
        this.num = num;
    }
    public async generate ( ) : Promise<number[]> {
        if( this.proxy == null ){
            const factory = Comlink.wrap<PrimeNumberClassConstructors>(this.worker);
            this.proxy = await new factory(this.num);
        }
        return this.proxy.generate()
    }

    public async dispose() {
        if( this.proxy ) this.proxy[Comlink.releaseProxy]();
        this.worker.terminate();
    }

}

Notice the this.worker.terminate(); as pointed out in #31 i think it is necessary.