Multithreading Redux Actions

[nelsonni]

Describe the bug Loading large files into the Redux store results in significant delays which trigger the Redux Serializability Middleware threshold of 32ms per event (with most events requiring 30-40ms, but occasionally spiking above 60ms). These events correspond to an unresponsive UI and sluggish load times.

For larger projects (with lots of files), these warnings will be triggered every time the FSCache subsystem attempts to update the Redux store state. An example warning message (typically one of 100+ such messages that appear in the DevTools console logs when loading a large project such as EPICLab/ant-design):

SerializableStateInvariantMiddleware took 40ms, which is more than the warning threshold of 32ms. 
If your state or actions are very large, you may want to disable the middleware as it might cause too much of a slowdown in development mode. See https://redux-toolkit.js.org/api/getDefaultMiddleware for instructions.
It is disabled in production builds, so you don't need to worry about that. 
    at FileComponent (webpack-internal:///./src/components/Explorer/FileComponent.tsx:47:26)
    at ul
    at TreeView (webpack-internal:///./node_modules/@material-ui/lab/esm/TreeView/TreeView.js:57:24)
    at WithStyles (webpack-internal:///./node_modules/@material-ui/styles/esm/withStyles/withStyles.js:64:31)
    at div
    at Explorer (webpack-internal:///./src/components/Explorer/Explorer.tsx:16:49)
    at Content (webpack-internal:///./src/components/Card/ContentFront.tsx:15:18)
    at div
    at ContentFront
    at Transition (webpack-internal:///./node_modules/react-transition-group/esm/Transition.js:132:30)
    at CSSTransition (webpack-internal:///./node_modules/react-transition-group/esm/CSSTransition.js:119:35)
    at div
    at CardComponent (webpack-internal:///./src/components/Card/CardComponent.tsx:67:56)
    at div
    at Canvas (webpack-internal:///./src/components/Canvas/Canvas.tsx:59:51)
    at FSCacheProvider (webpack-internal:///./src/store/cache/FSCache.tsx:55:28)
    at DndProvider (webpack-internal:///./node_modules/react-dnd/dist/cjs/core/DndProvider.js:40:11)
    at PersistGate (webpack-internal:///./node_modules/redux-persist/es/integration/react.js:38:5)
    at Provider (webpack-internal:///./node_modules/react-redux/es/components/Provider.js:15:3)
    at App (webpack-internal:///./src/app.tsx:43:27)

To Reproduce Steps to reproduce the behavior:

1. Launch Synectic
2. Go to File → Open...
3. Select the root directory of a project containing large files
4. See warnings in the Developer Tools console

Expected behavior Loading files in the background and presenting necessary details in the UI as the user opens files, directories, and branches in the different cards displayed on the canvas.

Screenshots

Versions (please complete the following information):

• OS: Linux, MacOS, Windows
• Synectic Version: 3.2.0

Additional context Attempts to load EPICLab/ant-design result in the following console log warnings: localhost-1672852410238.log

[nelsonni]

Attempts to unload some of the CPU intensive calls (i.e. fetchRepo, fetchBranches, updateVersionedMetafile) to background threads inevitably result in a catch-22 situation. For example, in order to use Web Workers in a Node.js environment using webpack, you must use ESM per webpack docs:

Similar syntax is supported in Node.js (>= 12.17.0):

import { Worker } from 'worker_threads';

new Worker(new URL('./worker.js', import.meta.url));

Note that this is only available in ESM. Worker in CommonJS syntax is not supported by either webpack or Node.js.

But attempts to run this code in Electron result in the following:

This is because Electron currently only supports CommonJS modules. However, there are some potential future fixes that might introduce ESM support in Electron in the future:

• electron/electron#21457
• electron/electron#37535

[nelsonni]

Other multi-threading options include:

• Attempting to use Electron's utilityProcess to create child processes with Node.js and Message ports enabled. It provides the equivalent of child_process.fork API from Node.js but instead uses Services API from Chromium to launch the child process. Although, this option requires resolving some of the restrictions placed on these processes under the Electron Context Isolation feature.

• Using electron-redux@alpha which provides a Redux Store Enhancer that helps loosely synchronize the redux stores in multiple Electron processes. The use case for this module is described as:

  When working with Electron, using Redux poses couple of problems, since main and renderer processes are isolated and IPC is a single way of communication between them. This library, enables you to register all your Redux stores in the main & renderer process, and enable cross-process action dispatching & loose store synchronization.

  However, the v2 alpha branch throws errors when attempting to launch Synectic and only the older v1 is able to startup. But the v1 version has a major sticking point, which is outlined in their The road to electron-redux 2.0 issue, and that is that v1 still uses Electron's remote dependency. The remote module was deprecated in Electron 14.0, which was released on 2021-Aug-31 and reached EOL on 2022-Mar-29; which is a significantly older version given that the current version is 25.0.0 as of 2023-May-30.

• There is redux-state-sync which is a lightweight Redux Middleware for syncing your redux state across browser tabs. It uses the Electron Broadcast Channel API to listen and dispatch exactly the same actions that were dispatched in other tabs to keep the redux state in sync. Unfortunately, this library requires modifying the Redux-Toolkit built-in reducers provided by createEntityAdapter as a workaround described in aohua/redux-state-sync#124.

[nelsonni]

Resolved in Synectic v4.0.0.