worker_threads in Main (!) process (*not Web worker in render, but node's worker_threads)

[b-karin]

I've seen a lot of issues related to workers and there is a misunderstanding between the two concepts: Web Workers and worker_threads: one is related to the render process, while another is to the main one. Some state that worker_threads DO NOT WORK in Electron: Notes: Don't try to use node's native worker_threads, they don't work in Electron But it's not true, they do not work in the render process, however, they do work in the main and you can easily communicate through parentPort.postMessage in worker and worker.on in the background.js (for example, but I encourage to separate the concerns and do not keep everything in one file) and then through IPC.

There are enough solutions for using Web Workers. One of them is in mentioned above link

As for working_threads, there are none. The way I manage it to work:

/* eslint-disable import/prefer-default-export, import/no-webpack-loader-syntax */
/* global __static */
import { Worker } from 'worker_threads';
import path from 'path';

import uartWorker from 'raw-loader!./uart.worker';

const isDevelopment = process.env.NODE_ENV !== 'production';

const appFolder = path.join(__static, '../');

const uart = new Worker(
  isDevelopment
    ? path.join(appFolder, 'src/main/threads/uart.worker.js')
    : uartWorker,
  {
    eval: !isDevelopment,
    workerData: {
      path: isDevelopment ? appFolder : process.resourcesPath,
    },
  },
);

export {
  uart,
};

What is bad about this solution that despite 'uart.worker.js' gets bundled with the app - it's not bundled itself, so without adding dependencies to externals or unpacking them dependencies won't work.

I see two better solutions, but I need some help. Looking ahead, Worker-plugin is only for web workers.

There is Threads.js and its documentation states it does work with worker_threads. It has its own loader threads-plugin built on top of worker-plugin As @nklayman suggests configureWebpack only effects the renderer process, while the electronBuilder.externals array is set for both processes (and sets other required configuration).

So I tried to set electronBuilder.externals.plugins = [new ThreadsPlugin()], while using

import { Worker } from 'threads';
const uart = new Worker('./uart.worker.js');
...

The desired outcome was a separate bundle for uart.worker.js that had to be unpacked and resolved itself (as threads docs state) So the first question is it a problem with my vue.config.js configuration and ThreadsPlugin is simply not used or a problem with a loader itself?

My second solution is not based on any external plugins, but I need some help with the config. So I think I should create separate bundles for each thread and then referencing them in new Worker(). With regular Webpack I could simply achieve it by setting multiple entry points. The question is: what is a proper way of creating an additional entry point within vue.config.js?

chainWebpackMainProcess: (config) => {
        config.entry('uart.worker')
          .add('./src/main/threads/uart.worker.js')
          .end();

works and acts the same as adding it to preload.

 preload: {
        preload: 'src/preload.js',
        'uart.worker': 'src/main/threads/uart.worker.js'
      },

In both cases, we can then use asarUnpack and reference unpacked files in new Worker();

new Worker(path.join(__static, '../app.asar.unpacked/uart.worker.js'));

And the final solution to make it work on both dev (serve) and production (build):

/* eslint-disable import/prefer-default-export, import/no-webpack-loader-syntax */
import { Worker } from 'worker_threads';
import path from 'path';

const isDevelopment = process.env.NODE_ENV !== 'production';

const uart = new Worker(path.join(__dirname,
  isDevelopment
  ? 'uart.worker.js'
  : '../app.asar.unpacked/uart.worker.js'));

export {
  uart,
};

I think if to define the most correct way of adding additional entry points (e.g. should it be with preload or in chainWebpackMainProcess: (config) => { config.entry().add ... } it could be documented on guide page

[a1156883061]

There is my config to use Thread.js In vue.config.js Set chainWebpackMainProcess config and use ThreadsPlugin, and set target: 'electron-node-worker' for it, and don't ignore tiny-worker, because in my App, it cause build error.

const ThreadsPlugin = require('threads-plugin');
...
pluginOptions: {
    electronBuilder: {
      builderOptions: {
        // you can find this file in 'bundled' folder , this is file that your worker using
        asarUnpack: ['0.worker.js'],
      },
      chainWebpackMainProcess: (config) => {
        // Chain webpack config for electron main process only
        config
          .plugin('thread')
          .use(ThreadsPlugin, [{ target: 'electron-node-worker' }]);
        // config.plugin('ignorePlugin').use(IgnorePlugin, [
        //   {
        //     resourceRegExp: /(tiny-worker)/,
        //   },
        // ]);
      },
      // externals: ['0.worker.js'],
    },
  },

And my worker code is like bellow :

import { spawn, Thread, Worker } from 'threads';

async function sortArrayByWorker(dirName: string, array: string[]) {
  const sortWorker = await spawn<{
    sortArray: (dirPath: string, files: string[]) => string[];
  }>(new Worker('../worker/sort-array.ts', { type: 'module' }));
  const result = await sortWorker.sortArray(dirName, array);
  await Thread.terminate(sortWorker);
  return result;
}

export default sortArrayByWorker;```

[MichaelJCole]

I used this code to solve the packaging issue by bundling everything through async imports.

It's been a couple years, any more thoughts on this? I was surprised node workers weren't included in the documentation.

I haven't tested on more than linux. I'm using electron-vite with electron 28 and esm modules.

// index.ts - this is the entry point for electron-vite
import { isMainThread } from 'worker_threads'
import 'better-sqlite3-multiple-ciphers' // for packaging
import { fileURLToPath } from 'url'

const filename = fileURLToPath(import.meta.url)  // pass path to this file to main() so it can start this file as a worker

if (isMainThread) {
  console.log('STARTING MAIN')
  const { main } = await import('@main/main')
  main(filename)
} else {
  console.log('STARTING WORKER')
  const { worker } = await import('@main/worker')
  worker()
}

// main.ts - this is the main electron node process
import { configureApp } from '@main/app'
import { configureIPC } from '@main/ipc'
import { Worker } from 'worker_threads'

export async function main(filename: string) {
  await configureApp().ready
  configureIPC()
  startWorker(filename)
}

function startWorker(filename: string) {
  const worker = new Worker(filename, {
    workerData: 'dataToPassToNewThread'
  })

  worker.on('message', (data) => {
    console.log('WORKER SAYS:', data)
  })
  worker.on('error', (err) => {
    console.log('WORKER ERROR:', err)
  })
  worker.on('exit', (code) => {
    console.log('WORKER EXITED WITH CODE:', code)
  })
}

// worker.ts - this is the worker
import { parentPort, workerData } from 'worker_threads'

export function worker() {
  console.log('STARTING WORKER')
  const data = workerData
  parentPort?.postMessage(`You said "${data}".`)
}