taskforcesh / bullmq

BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis
https://bullmq.io
MIT License
5.89k stars 380 forks source link

sandboxed worker .cjs #463

Open andrisi opened 3 years ago

andrisi commented 3 years ago

Trying to use a sandboxed processor file from a project that uses ES6 modules, but as BullMQ seems to use require(), this does not work with ES6 modules in nodejs 15. require() says "require() of ES modules is not supported". So I'd use a CommonJS modul, but that need a .cjs extension, and it's not in the list of supported extensions in your worker.js around line 52. If I add .cjs it starts but I'm afraid I'll still not be able to use code from the rest of my project as it's a commonJs module now...

const supportedFileTypes = ['.js', '.ts', '.flow'];

andrisi commented 3 years ago

changing tha handling of the 'init' message in your worker.js as this helps:

            import(msg.value).then((mod) => {

                if (mod.default) {
                    // support es2015 module.
                    processor = mod.default;
                }
                if (processor.length > 1) {
                    processor = util_1.promisify(processor);
                }
                else {
                    const origProcessor = processor;
                    processor = function () {
                        try {
                            return Promise.resolve(origProcessor.apply(null, arguments));
                        }
                        catch (err) {
                            return Promise.reject(err);
                        }
                    };
                }
                status = 'IDLE';
                process.send({
                    cmd: 'init-complete',
                });

            })
andrisi commented 3 years ago

Seems to be a minor issue - and hope of this getting fixed. Sandboxed processors would be so much more useful.

jstayton commented 3 years ago

There's a PR in Bull for this same functionality: https://github.com/OptimalBits/bull/pull/2085

Unfortunately, it'll probably be a breaking change here as well and require a new major version. Maybe when support for Node.js v10 is dropped?