search

ipfs / js-ipfs

IPFS implementation in JavaScript
https://js.ipfs.tech
Other
7.44k stars 1.25k forks source link

ipfs-http-client breaks Webpack builds #3452

Closed dapplion closed 1 year ago

dapplion commented 3 years ago

With 48.1.2 a server webpack breaks with (error message below). See CI run https://github.com/dappnode/DNP_DAPPMANAGER/pull/516/checks?check_run_id=1563460710#step:3:279

webpack.config.js

const path = require("path");
const { NODE_ENV = "production" } = process.env;
module.exports = {
  entry: "./src/index.ts",
  mode: NODE_ENV,
  target: "node",
  output: {
    path: path.resolve(__dirname, "build"),
    filename: "index.js"
  },
  // externals: [/node_modules/, "bufferutil", "utf-8-validate"],
  resolve: {
    extensions: [".ts", ".js"]
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        use: ["ts-loader"]
      }
    ]
  },
  optimization: {
    // Minimization does not provide great disk space savings, but reduces debug capacity
    minimize: false
  },
  devtool: "source-map"
};

webpack error

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,26)
      TS1005: ',' expected.

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,61)
      TS1005: ',' expected.

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,151)
      TS1005: ',' expected.

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,180)
      TS1005: ',' expected.

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,215)
      TS1005: ',' expected.

+ many more lines with the same error
dapplion commented 3 years ago

It also breaks with a different error

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/electron-fetch/lib/index.es.js
Module not found: Error: Can't resolve 'electron' in '/home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/electron-fetch/lib'
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/electron-fetch/lib/index.es.js 1262:13-32
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-utils/src/fetch.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-utils/src/http/fetch.browser.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-utils/src/http/fetch.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-utils/src/http.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/src/lib/core.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/src/lib/configure.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/src/version.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/src/index.js
 @ ./src/modules/ipfs/ipfsSetup.ts
 @ ./src/modules/ipfs/methods/pinAdd.ts
 @ ./src/ethForward/index.ts
 @ ./src/index.ts
achingbrain commented 3 years ago

So we can replicate the issue, what version of webpack is this?

dapplion commented 3 years ago

https://github.com/dappnode/DNP_DAPPMANAGER/blob/60ac93345872a9b8f66bd1ac63b7b620dddabe65/packages/dappmanager/package.json#L91

"webpack": "^4.43.0",
"webpack-cli": "^3.3.11"
TheoXD commented 3 years ago

I'm in the same boat, building with ipfs-http-client just spams the console with "',' expected" errors.

perfectmak commented 3 years ago

I just experienced this now. Is there any update on how to fix it?

I was using ipfs-http-client version 49.0.2. I had to downgrade to 47.0.0 and I stopped getting the error.

NickBeukema commented 3 years ago

I've also experienced this issue when deploying a function into AWS Lambda, Webpack errors with the following issue:

ERROR in /build/node_modules/electron-fetch/lib/index.es.js
Module not found: Error: Can't resolve 'electron' in '/build/node_modules/electron-fetch/lib'
 @ /build/node_modules/electron-fetch/lib/index.es.js 1262:13-32
 @ /build/node_modules/ipfs-utils/src/fetch.js
 @ /build/node_modules/ipfs-utils/src/http/fetch.browser.js
 @ /build/node_modules/ipfs-utils/src/http/fetch.js
 @ /build/node_modules/ipfs-utils/src/http.js
...

I'm somewhat concerned that ipfs-http-client ends up pulling in Electron through ipfs-utils, am I interpreting that correctly?

Is the suggested fix for this to downgrade?

jthomerson commented 2 years ago

I have the same problem. Then when I installed a couple different versions (47 and 55), I got "Module "stream" has been externalized for browser compatibility and cannot be accessed in client code."

achingbrain commented 2 years ago

@jthomerson stream is a node js module and webpack doesn't polyfill them since version 5. If your project build is pulling it in you need to make it available somehow.

There are examples of bundling ipfs and ipfs-http-client using webpack here: https://github.com/ipfs-examples?q=webpack

jthomerson commented 2 years ago

@achingbrain thanks. I probably should've clarified a bit. I'm aware of stream being for node. What I don't understand is why it's being pulled in. I assumed it was a problem with ipfs-http-client itself, especially since I had the same Electron-centric error originally posted in this bug. But the problem may be the bundler itself. I'm basically following this React-based tutorial, but instead of using React, I started the project with Vitesse - so Vue for the framework, Vite for the bundler.

If anyone happens to have any ideas on why Vite might be pulling in Stream, I'd be happy to hear them. All I'm doing in the code is this:

import { create as ipfsHttpClient } from 'ipfs-http-client'

const client = ipfsHttpClient('https://ipfs.infura.io:5001/api/v0')

const added = await client.add(
  file,
  {
    progress: prog => console.log(`received: ${prog}`),
  },
)
const url = `https://ipfs.infura.io/ipfs/${added.path}`
item.value.fileURL = url
achingbrain commented 2 years ago

I'm unfamiliar with Vite, but it looks like it uses Rollup under the hood and not webpack so I don't think it's related to this issue.

ShravanSunder commented 2 years ago

This is an additional problem with any bundler that requires esm modules. It looks like ipfs-http-client is requiring electron and electron-fetch for some reason in versions after 47.

ShravanSunder commented 2 years ago

You should be able to work around this by marking electron and electron-fetch as external in webpack

I've managed to replicate and work around this in vite using this plugin and naming electron and electron-fetch as externals

milahu commented 2 years ago

I started the project with Vitesse - so Vue for the framework, Vite for the bundler.

related, since vite is similar to rollup

SgtPooki commented 1 year ago

js-ipfs is being deprecated in favor of Helia. You can https://github.com/ipfs/js-ipfs/issues/4336 and read the migration guide.

Please feel to reopen with any comments by 2023-06-02. We will do a final pass on reopened issues afterward (see https://github.com/ipfs/js-ipfs/issues/4336).

Unfortunately, this won't be resolved in js-ipfs repos, but Helia should not run into these problems. If you're still on CJS, you will likely be between a rock and a hard place, but I highly recommend updating your package to ESM and using vite/rollup bundling to solve cjs/esm pains.