taskforcesh / bullmq

BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis
https://bullmq.io
MIT License
6.14k stars 402 forks source link

[Bug]: Critical dependency: the request of a dependency is an expression #1759

Open rolznz opened 1 year ago

rolznz commented 1 year ago

Version

v3.10.2

Platform

NodeJS

What happened?

When I import from bullmq in a NextJS project I get the following message in the console output:

Critical dependency: the request of a dependency is an expression

Import trace for requested module: ./node_modules/bullmq/dist/cjs/classes/child-processor.js ./node_modules/bullmq/dist/cjs/classes/index.js ./node_modules/bullmq/dist/cjs/index.js

How to reproduce.

  1. Create a new NextJS project (https://nextjs.org/)
  2. Add bullmq as a dependency
  3. import { Queue, QueueOptions } from "bullmq";

Relevant log output

Critical dependency: the request of a dependency is an expression

Import trace for requested module:
./node_modules/bullmq/dist/cjs/classes/child-processor.js
./node_modules/bullmq/dist/cjs/classes/index.js
./node_modules/bullmq/dist/cjs/index.js

Code of Conduct

manast commented 1 year ago

This is happening with external processors right? I think the easiest, for now, would be to use normal processors instead, as I am not sure this can be resolved at all as we need to be able to dynamically load the processor in order to run it in a separate process.

manast commented 1 year ago

If it is happening regardless then it must be some kind of pre-processing NextJS that detects this possibility, maybe it can be disabled or something? I am not familiar with NextJS at all unfortunatelly.

rolznz commented 1 year ago

@manast how would I go about using the normal processors? I am just importing { Queue, QueueOptions } from bullmq

adicco commented 1 year ago

@rolznz Use serverComponentsExternalPackages in next.config.js, e.g.:

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  experimental: {
    serverComponentsExternalPackages: ['bullmq']
  }
}

module.exports = nextConfig
theDanielJLewis commented 1 year ago

I was just having this problem, too. But adding @adicco's suggestion seems to have fixed it for me.

I'm also running Next.js 13 with BullMQ to handle queues. Most of the queue stuff is done on the backend running under a separate Node process, but the frontend can add something to the queue, too. It was working fine, and now that error message is gone.

Thank you!

tflanagan commented 1 year ago

@adicco's suggestion did not work for me.

This issue persists even with the config change.

murderteeth commented 11 months ago

for me, @adicco 's suggestion gets rid of the Critical dependency message, but my nextjs api endpoint continues to 404 when including any bullmq deps. in my case, i'm using apollo graphql to provide clients with stats on various queues among other things via a next api endpoint. haven't figured out a way around this.. there's no error messages anymore lol

m1daz commented 7 months ago

Any updates on a permanent fix to this? I am encountering this on latest NEXT version and was wondering as to why, since on 14.0.3 I never saw this error.

somersby10ml commented 7 months ago

I use express.js + webpack5 and the same problem occurs.

stepan-twnty commented 3 months ago

@manast any updated?

I have the same problem. Next JS 14.2.0 imports Queue via an instrumentation file.

manast commented 3 months ago

@stepan-twnty no I haven't. I do not know how to solve this issue unfortunately as I am not very familiar with the NestJS ecosystem.

stepan-twnty commented 3 months ago

Thx for the quick response @manast. I fixed it by adding @adicco's suggestion.

I went through the Next JS documentation and checked what does mean "serverComponentsExternalPackages".

So if I understand it correctly Next JS by default automatically bundles all imports inside <root>/pages/api folder to the client bundle and BullMQ or some of its dependencies use Commons JS's "require" feature which is not supported on client bundle. By using "serverComponentsExternalPackages" we remove bullmq from the client bundle and only include it on the server bundle.