Cannot find module './fetch.node'

jwkim960315 commented 3 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

SDK version

4.2.0

Lit Network

cayenne

Description of the bug/issue

Hi! Following the installation of the package (having attempted both the nodejs-exclusive and isomorphic versions), I attempted to initialize a lit node client, which resulted in the following error:

Import trace for requested module:
./node_modules/ipfs-utils/src/http/fetch.js
./node_modules/ipfs-utils/src/http.js
./node_modules/ipfs-http-client/cjs/src/files/rm.js
./node_modules/ipfs-http-client/cjs/src/files/index.js
./node_modules/ipfs-http-client/cjs/src/index.js
./node_modules/@lit-protocol/encryption/src/lib/encryption.js
./node_modules/@lit-protocol/encryption/src/index.js
./node_modules/@lit-protocol/lit-node-client-nodejs/src/index.js
./app/api/lit/route.ts
 ⨯ Error: Cannot find module './fetch.node'
    at webpackEmptyContext (/workspaces/Allegro/allegro/.next/server/app/api/lit/route.js:22:10)
    at eval (webpack-internal:///(rsc)/./node_modules/ipfs-utils/src/http/fetch.js:14:118)
    at (rsc)/./node_modules/ipfs-utils/src/http/fetch.js (/workspaces/Allegro/allegro/.next/server/vendor-chunks/ipfs-utils.js:70:1)
    at __webpack_require__ (/workspaces/Allegro/allegro/.next/server/webpack-runtime.js:33:43)
    at eval (webpack-internal:///(rsc)/./node_modules/ipfs-utils/src/http.js:2:37)
    at (rsc)/./node_modules/ipfs-utils/src/http.js (/workspaces/Allegro/allegro/.next/server/vendor-chunks/ipfs-utils.js:50:1)
    at __webpack_require__ (/workspaces/Allegro/allegro/.next/server/webpack-runtime.js:33:43)
    at eval (webpack-internal:///(rsc)/./node_modules/ipfs-http-client/cjs/src/lib/core.js:9:12)
    at (rsc)/./node_modules/ipfs-http-client/cjs/src/lib/core.js (/workspaces/Allegro/allegro/.next/server/vendor-chunks/ipfs-http-client.js:770:1)
    at __webpack_require__ (/workspaces/Allegro/allegro/.next/server/webpack-runtime.js:33:43)
    at eval (webpack-internal:///(rsc)/./node_modules/ipfs-http-client/cjs/src/lib/configure.js:5:12)
    at (rsc)/./node_modules/ipfs-http-client/cjs/src/lib/configure.js (/workspaces/Allegro/allegro/.next/server/vendor-chunks/ipfs-http-client.js:760:1)
    at __webpack_require__ (/workspaces/Allegro/allegro/.next/server/webpack-runtime.js:33:43)
    at eval (webpack-internal:///(rsc)/./node_modules/ipfs-http-client/cjs/src/bitswap/wantlist.js:6:17)
    at (rsc)/./node_modules/ipfs-http-client/cjs/src/bitswap/wantlist.js (/workspaces/Allegro/allegro/.next/server/vendor-chunks/ipfs-http-client.js:80:1) {
  code: 'MODULE_NOT_FOUND',
  page: '/api/lit'
}

I looked at /node_modules/ipfs-utils/src/http/fetch.js and found the following code:

let implName = './fetch.node'

if (typeof XMLHttpRequest === 'function') {
  // Electron has `XMLHttpRequest` and should get the browser implementation
  // instead of node.
  implName = './fetch.browser'
}

/** @type {fetchImpl} */
const fetch = require(implName)

module.exports = fetch

I understand the issue stems from ipfs-utils, but is there a workaround for this?

Severity of the bug

High, since I cannot even initialize the lit node client

Steps To Reproduce

Create Next@13.5.6 app
Install @lit-protocol/lit-node-client-nodejs
Create a folder (ex. lit) under app/api directory and route.ts file

Initialize lit node client as such:

export async function GET() {
const litNodeClient = new LitJsSdk.LitNodeClientNodeJs({
litNetwork: "cayenne",
});
}

Call GET /api/lit from page.tsx

Link to code

No response

Anything else?

No response

0x62 commented 2 months ago

I'm also hitting this issue with a Remix app, there is also an open issue on the ipfs-utils library for it. ipfs-http-client has been deprecated for a while, so won't be updated to fix this anytime soon.

Can Lit move to something like https://github.com/ipfs/helia-verified-fetch instead?

jrhe commented 2 months ago

Spent a fair bit of time looking at this today as it's blocking me developing anything with lit.

ipfs-http-client is just used by the encryptToIpfs method. The decryptFromIpfs uses vanilla fetch already, so it might make sense use fetch for encrypt too and remove a dependency. If a client library is desired js-kubo-rpc-client appears to be a drop in replacement, but it suffers the same issue, so it would need changing.

Encrypt uses Infura's gateway and decrypt uses pinata's gateway - is there any particular reason for this? It might be best to allow the user to specify the gateway and params. If a gateway is down they may want to use another. Case in point, Infura are not currently issuing new API keys for it's IPFS gateway (Temporary limits on new IPFS Keys).

If js-kubo-rpc-client'is used, the encrypt/decrypt methods could be adapted to accept a client object as a parameter, and the encrypt/decrypt methods could call that.

Ansonhkg commented 2 months ago

@jrhe @0x62 We will have this fixed in the next day or 2, @MaximusHaximus has been working on it! :)

johnnynanjiang commented 2 months ago

Another similar issue @Ansonhkg @MaximusHaximus

yarn nx run react:serve failed with error [react]: > NX Unable to resolve @nx/web:dev-server.

jrhe commented 2 months ago

Also, both isomorphic and node packages patch globalthis.fetch. Do we need this given fetch was added to Node 18, and is stable as of Node 21 or do you want to support older versions than this? I expect this might create issues as other packages also patch globalthis.fetch. In particular, I know Next.js patches it, which might be quite commonly used with the lit sdk.

MaximusHaximus commented 2 months ago

Also, both isomorphic and node packages patch globalthis.fetch. Do we need this given fetch was added to Node 18, and is stable as of Node 21 or do you want to support older versions than this? I expect this might create issues as other packages also patch globalthis.fetch. In particular, I know Next.js patches it, which might be quite commonly used with the lit sdk.

Hey all, just a brief update:

We're replacing our own internal polyfilling of fetch with cross-fetch to ensure it is handled properly across multiple environments and versions of node.
We're removing the ipfs-http-client dependency entirely, as it was only used by one convenience method and the amount of code behind it is quite huge, even apart from it being deprecated :). This issue will be resolved by #427 . Sorry about the delay on getting this fixed up -- it turned into a bit of a rabbit hole -- and thanks so much for your patience 💖 .

Ansonhkg commented 2 months ago

Hey @johnnynanjiang @jrhe @jwkim960315 we have just published v5.0.0. Let us know if you're still seeing the above issues! :) https://github.com/LIT-Protocol/js-sdk/pull/432

johnnynanjiang commented 2 months ago

@Ansonhkg the following 2 still not working, I expect to run test apps to see how Lit Protocol works, but they seem to be low priority. If they don't work, how builders expect to see how Lit Protocol works?

// react
yarn nx run react:serve

error
 >  NX   Unable to resolve @nx/web:dev-server.

and

// nodejs
yarn nx run nodejs:serve

error
Cannot find module '../Lit'

LIT-Protocol / js-sdk