[Bug] Third party import of Deck.gl library, build problems with next.js

geoerika commented 1 month ago

Description

I have a react library of components that has a submodule for deck.gl-based map components. Let's call it @complibrary/maps.

My aim is to use this external map library in a next.js app. Let's call it: mainapp. I installed the map library in the mainapp, however, during the build I am getting the following error:

Failed to compile.

./node_modules/@complibrary/maps/dist/index.js
Module not found: Can't resolve 'worker_threads'

./node_modules/@complibrary/maps/dist/index.mjs
Module not found: Can't resolve 'worker_threads'

Flavors

[ ] Script tag
[X] React
[ ] Python/Jupyter notebook
[ ] MapboxOverlay
[ ] GoogleMapsOverlay
[ ] CartoLayer
[ ] ArcGIS

Expected Behavior

I am expecting no build errors.

Steps to Reproduce

Unfortunately, all my work is in private repos. In my dependency list for the next.js app, I have deck.gl & the map library installed:

"dependencies": {
    "@complibrary/maps": "0.1.0",
    "deck.gl": "^9.0.27",
    "next": "^14.2.3",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
  },

If I add the code below to the next.config file, the build error is solved, however the map loses interactivity: no zoom, pan, etc.

 webpack: (config, { isServer }) => {
    if (!isServer) {
      config.resolve.fallback = {
        worker_threads: false
      }
    }
    return config;
  },

Environment

Framework version: deck.gl@9.0.x
Browser: Safari, Chrome, possibly others
OS: MacOS

Logs

No response

ibgreen commented 1 month ago

The code causing this issue is most likely in a loaders.gl dependency, but don't really know enough what is under the hood of Next.js (what bundler it is using etc) to have a clue on how to address it. I can say that managing the worker_threads dependency so that it only imports in Node.js has been tricky. You are welcome to engage in loaders.gl discussions tab if you can find more information or have suggestions.

FWIW, I don't see the issue in an internal Next.js application I have access to, and it is using exactly the same version of Next.js but it is stlll on deck.gl v8 so maybe will be able to look more hands on at this down the road once the deck upgrade happens.

geoerika commented 1 month ago

Yes, the issue is not with a direct usage of deck.gl in the next.js app. The same map component included in the next.js app works perfectly fine.

geoerika commented 1 month ago

I also suspect the issue resides in the loaders.gl package. I noticed some similar errors occurred a few years back and have been solved with loaders.gl changes.

https://github.com/visgl/deck.gl/issues/6906

donmccurdy commented 3 weeks ago

I've taken an approach in the past that required four total builds:

browser, CJS
browser, ESM
node, CJS
node, ESM

As seen in ndarray-pixels / package.json. The same could likely work here, with the caveat that I have little experience with Server Side Components (SSC) and can't really speak to those. For a few reasons, this may be easier if/when we are able to publish ESM-only packages in some unspecified future version.

ibgreen commented 3 weeks ago

I've taken an approach in the past that required four total builds:

Yes. FWIW I've had serious problems working using libraries that have node-specific builds. It depends on the build setup once has in place and how customized it is. In my view it is a case of "pick your poison". I hold out some hope that this is fixable without going there.

visgl / deck.gl