Webpack+Typescript project doesn't build - Esm export errors

sebastiendan commented 3 years ago

Version: ≥ 0.56.0 (e.g. 0.58.4)
Platform: Mac

Severity:

High - The main functionality of the application does not work, API breakage, repo format breakage, etc.

Description:

After adding ipfs to my Webpack+Typscript project, I cannot build it anymore due to a very large number of esm related errors. Here's one of them:

ERROR in ./node_modules/ipns/esm/src/index.js 112:15-27
export 'decode' (imported as 'cborg') was not found in 'cborg' (possible exports: __esModule, create, createWithExpiration, embedPublicKey, extractPublicKey, getIdKeys, getLocalKey, marshal, namespace, namespaceLength, unmarshal, validate, validator)
 @ ./node_modules/ipfs-core/src/ipns/publisher.js 13:13-28
 @ ./node_modules/ipfs-core/src/ipns/index.js 10:22-44
 @ ./node_modules/ipfs-core/src/components/ipns.js 3:13-31
 @ ./node_modules/ipfs-core/src/components/index.js 23:16-33
 @ ./node_modules/ipfs-core/src/index.js 10:19-42
 @ ./node_modules/ipfs/src/index.js 7:22-42

I successfully built the webpack example from js-ipfs-examples so I assume the problem is coming from my typescript configuration. Here are all my config files:

package.json

``` { "name": "***", "version": "1.0.0", "description": "***", "main": "dist/index.js", "files": [ "dist" ], "types": "dist/index.d.ts", "repository": { "type": "git", "url": "***" }, "publishConfig": { "registry": "https://npm.pkg.github.com" }, "author": "Sébastien Dan", "license": "MIT", "scripts": { "build": "webpack --color", "lint": "eslint -f unix \"src/**/*.{ts,tsx}\"", "test": "NODE_OPTIONS=--experimental-vm-modules jest --detectOpenHandles", "test:coverage": "yarn test --coverage && jest-coverage-badges", "test:watch": "yarn test --watch", "watch": "yarn build --watch" }, "dependencies": { "@polkadot/api": "^5.2.1", "@polkadot/ui-keyring": "^0.85.1", "@polkadot/ui-settings": "^0.85.1", "aes-js": "^3.1.2", "eciesjs": "^0.3.11", "ipfs": "^0.58.4", "remove": "^0.1.5" }, "devDependencies": { "@babel/plugin-transform-modules-commonjs": "^7.15.0", "@types/jest": "^27.0.1", "@typescript-eslint/eslint-plugin": "^4.30.0", "@typescript-eslint/parser": "^4.30.0", "babel-jest": "^27.0.0-next.11", "crypto-browserify": "^3.12.0", "eslint": "^7.32.0", "eslint-plugin-tsdoc": "^0.2.14", "eslint-webpack-plugin": "^3.0.1", "jest": "^27.0.0-next.11", "jest-coverage-badges": "^1.1.2", "process": "^0.11.10", "resolve-typescript-plugin": "^1.1.1", "stream-browserify": "^3.0.0", "ts-jest": "27.0.5", "ts-loader": "^9.2.5", "typescript": "^4.3.5", "webpack": "^5.50.0", "webpack-cli": "^4.8.0" }, "jest": { "coverageDirectory": "coverage", "coverageProvider": "v8", "coverageReporters": [ "json-summary", "text", "lcov" ], "globals": { "ts-jest": { "useESM": true } }, "testEnvironment": "node", "preset": "ts-jest/presets/default-esm" } } ```

webpack.config.js

``` const path = require('path') const webpack = require('webpack') const ESLintPlugin = require('eslint-webpack-plugin') module.exports = { entry: './src/index.ts', mode: 'development', devtool: 'source-map', module: { rules: [ { test: /\.ts$/, use: 'ts-loader', exclude: /node_modules/, }, ], }, resolve: { extensions: ['.ts', '.js'], fallback: { crypto: require.resolve('crypto-browserify'), stream: require.resolve('stream-browserify'), }, }, output: { filename: 'index.js', path: path.resolve(__dirname, 'dist'), library: { name: 'toposJsMessenger', type: 'umd', }, }, plugins: [ new webpack.ProvidePlugin({ process: 'process/browser', }), new webpack.ProvidePlugin({ Buffer: ['buffer', 'Buffer'], }), new ESLintPlugin({ extensions: ['ts'], }), ], } ```

tsconfig.json

``` { "compilerOptions": { "module": "commonjs", "target": "es2015", "declaration": true, "sourceMap": true, "outDir": "./dist", "esModuleInterop": true, "moduleResolution": "node" }, "include": ["src/**/*"], "allowJs": true } ```

Note: I was able to build with ipfs@0.55.0 only.

Would you know what config is breaking the build?

welcome[bot] commented 3 years ago

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review. In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment. Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

"Priority" labels will show how urgent this is for the team.
"Status" labels will show if this is ready to be worked on, blocked, or in progress.
"Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

achingbrain commented 3 years ago

This is due to a patch release of ipfs-bitswap ported to ESM that should have gone out as a major.

It's been re-released as CJS and revved properly for ESM - please delete your node_modules folder and reinstall.

sebastiendan commented 3 years ago

I see. I do have way less errors now but still have a few, related to ipfs-unixfs-importer:

ERROR in ./node_modules/ipfs-unixfs-importer/esm/src/options.js 5:16-33
export 'murmur3128' (imported as 'murmur3128') was not found in '@multiformats/murmur3' (possible exports: __esModule, importer)
 @ ./node_modules/ipfs-unixfs-importer/esm/src/index.js 2:0-42 6:15-29
 @ ./node_modules/ipfs-core/src/components/add-all/index.js 3:21-52
 @ ./node_modules/ipfs-core/src/components/root.js 4:24-44
 @ ./node_modules/ipfs-core/src/components/index.js 30:16-33
 @ ./node_modules/ipfs-core/src/index.js 10:19-42
 @ ./node_modules/ipfs/src/index.js 7:22-42

achingbrain commented 3 years ago

Can you try installing the deps with npm? For some reason I can only repro this error if I use yarn to install the deps - further discussion here: https://github.com/webpack/webpack/discussions/14268

sebastiendan commented 3 years ago

I actually have the same errors building with npm.

npm version: 6.14.11
node version: v16.8.0
yarn version: 1.22.11

ipfs-unixfs-importer in package-lock.json

``` "ipfs-unixfs-importer": { "version": "9.0.6", "resolved": "https://registry.npmjs.org/ipfs-unixfs-importer/-/ipfs-unixfs-importer-9.0.6.tgz", "integrity": "sha512-FgzODqg4pvToEMZ88mFkHcU0s25CljmnqX2VX7K/VQDckiZIxhIiUTQRqQg/C7Em4uCzVp8YCxKUvl++w6kvNg==", "requires": { "@ipld/dag-pb": "^2.0.2", "@multiformats/murmur3": "^1.0.3", "bl": "^5.0.0", "err-code": "^3.0.1", "hamt-sharding": "^2.0.0", "interface-blockstore": "^1.0.0", "ipfs-unixfs": "^6.0.6", "it-all": "^1.0.5", "it-batch": "^1.0.8", "it-first": "^1.0.6", "it-parallel-batch": "^1.0.9", "merge-options": "^3.0.4", "multiformats": "^9.4.2", "rabin-wasm": "^0.1.4", "uint8arrays": "^3.0.0" }, "dependencies": { "bl": { "version": "5.0.0", "resolved": "https://registry.npmjs.org/bl/-/bl-5.0.0.tgz", "integrity": "sha512-8vxFNZ0pflFfi0WXA3WQXlj6CaMEwsmh63I1CNp0q+wWv8sD0ARx1KovSQd0l2GkwrMIOyedq0EF1FxI+RCZLQ==", "requires": { "buffer": "^6.0.3", "inherits": "^2.0.4", "readable-stream": "^3.4.0" } }, "buffer": { "version": "6.0.3", "resolved": "https://registry.npmjs.org/buffer/-/buffer-6.0.3.tgz", "integrity": "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==", "requires": { "base64-js": "^1.3.1", "ieee754": "^1.2.1" } } } }, ```

achingbrain commented 3 years ago

Are you deleting the node_modules folder and reinstalling when switching between npm and yarn?

I've put a minimal repro together here based on the OP's files and opened an issue against webpack, though it got converted to a discussion.

sebastiendan commented 3 years ago

I just tried your repo and face the same result than you (builds fine with npm, not with yarn).

I don't know what's different in my case...

achingbrain commented 3 years ago

This should be fixed by https://github.com/webpack/enhanced-resolve/issues/304

sebastiendan commented 3 years ago

@achingbrain From the latest version of js-ipfs I understand application devs should use the ipfs-core package instead of ipfs (actually it seems we have to since imported from ipfs Ipfs.create is undefined). Testing that out brings a lot of esm related issues from ipns, ipfs-repo, ipfs-core, and probably others.

ERROR in ./node_modules/ipns/esm/src/index.js 112:15-27
export 'decode' (imported as 'cborg') was not found in 'cborg' (possible exports: __esModule, create, createWithExpiration, embedPublicKey, extractPublicKey, getIdKeys, getLocalKey, marshal, namespace, namespaceLength, unmarshal, validate, validator)
 @ ./node_modules/ipfs-core-config/esm/src/utils/ipns.js 1:0-29 2:50-73 4:9-30
 @ ./node_modules/ipfs-core-config/esm/src/libp2p.browser.js 7:0-10:25 40:28-37 41:27-35
 @ ./node_modules/ipfs-core/esm/src/components/libp2p.js 10:0-78 95:52-71
 @ ./node_modules/ipfs-core/esm/src/components/network.js 2:0-43 19:25-37
 @ ./node_modules/ipfs-core/esm/src/components/index.js 43:0-39 57:35-42
 @ ./node_modules/ipfs-core/esm/src/index.js 1:0-63 9:22-34
 @ ./src/Api.ts 19:20-40
 @ ./src/index.ts 13:13-29

The error I get in the browser when using the create function from the ipfs (0.60.0) package instead of ipfs-core:

Unhandled Rejection (TypeError): (0, ipfs_1.create) is not a function. (In '(0, ipfs_1.create)()', '(0, ipfs_1.create)' is undefined)

rvagg commented 3 years ago

@sebastiendan can you have a quick look through https://github.com/ipld/js-dag-cbor/issues/37, particularly at the bottom, and perhaps update your toolchain? It seems that we're stuck with the busted version of enhanced-resolve not being fixed and the newer version not being picked up by older tools that use it.

sebastiendan commented 3 years ago

@rvagg Thanks for the tip! A webpack upgrade did fix the problem.

ipfs / js-ipfs

Webpack+Typescript project doesn't build - Esm export errors #3868

Severity:

Description: