search

facebook / metro

🚇 The JavaScript bundler for React Native
https://metrobundler.dev
MIT License
5.15k stars 613 forks source link

compose-source-maps failure after manual changes upgrade 0.70.5 => 0.71.3 (Task :app:createBundleReleaseJsAndAssets or manual scripts) #938

Open alexkrav opened 1 year ago

alexkrav commented 1 year ago

New Version

0.71.3

Old Version

0.70.5

Build Target(s)

android

Output of react-native info

System: OS: macOS 13.2.1 CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz Memory: 4.14 GB / 16.00 GB Shell: 5.8.1 - /bin/zsh Binaries: Node: 18.3.0 - /usr/local/bin/node Yarn: 1.22.19 - /usr/local/bin/yarn npm: 8.11.0 - /usr/local/bin/npm Watchman: 2022.05.30.00 - /usr/local/bin/watchman Managers: CocoaPods: 1.11.3 - /Users/alexkrav/.rbenv/shims/pod SDKs: iOS SDK: Platforms: DriverKit 22.2, iOS 16.2, macOS 13.1, tvOS 16.1, watchOS 9.1 Android SDK: API Levels: 23, 28, 29, 30, 31, 33 Build Tools: 27.0.3, 28.0.3, 29.0.2, 29.0.3, 30.0.2, 30.0.3, 31.0.0, 31.0.0, 33.0.0 System Images: android-27 | Google Play Intel x86 Atom, android-28 | Google APIs Intel x86 Atom, android-29 | Google APIs Intel x86 Atom, android-29 | Google Play Intel x86 Atom, android-30 | Google APIs Intel x86 Atom, android-30 | Google Play Intel x86 Atom, android-31 | Google APIs Intel x86 Atom_64, android-33 | Google APIs Intel x86 Atom_64 Android NDK: 21.4.7075529 IDEs: Android Studio: 2021.3 AI-213.7172.25.2113.9123335 Xcode: 14.2/14C18 - /usr/bin/xcodebuild Languages: Java: 11.0.15 - /usr/bin/javac npmPackages: @react-native-community/cli: Not Found react: 18.2.0 => 18.2.0 react-native: 0.71.3 => 0.71.3 react-native-macos: Not Found npmGlobalPackages: react-native: Not Found

Issue and Reproduction Steps

After manual changes using Upgrade Helper android build task :app:createBundlePreReleaseJsAndAssets failed on creation of compose map

> Task :app:createBundlePreReleaseJsAndAssets
warn Package @react-native-community/google-signin contains invalid configuration: "dependency.platforms.ios.project" is not allowed. Please verify it's properly linked using "react-native config" command and contact the package maintainers about this.
warn Package @sentry/react-native contains invalid configuration: "dependency.platforms.ios.sharedLibraries" is not allowed,"dependency.hooks" is not allowed. Please verify it's properly linked using "react-native config" command and contact the package maintainers about this.
warn Package react-native-sqlite-storage contains invalid configuration: "dependency.platforms.ios.project" is not allowed. Please verify it's properly linked using "react-native config" command and contact the package maintainers about this.
warning: the transform cache was reset.
                Welcome to Metro v0.73.7
              Fast - Scalable - Integrated

Waiting for Watchman `query` (10s)...
Watchman `query` finished after 11.6s.
transform[stderr]: Error: infinite loop while processing mergePaths plugin.
transform[stderr]: Error: infinite loop while processing mergePaths plugin.
info Writing bundle output to:, /Users/alexkrav/TEST/neutron-0.71.x/android/app/build/ASSETS/createBundlePreReleaseJsAndAssets/index.android.bundle
info Writing sourcemap output to:, /Users/alexkrav/TEST/neutron-0.71.x/android/app/build/intermediates/sourcemaps/react/preRelease/index.android.bundle.packager.map
info Done writing bundle output
info Done writing sourcemap output
info Copying 1136 asset files
info Done copying assets
...
Error: Invalid mapping: {"generated":{"line":1,"column":1023478},"source":"/Users/alexkrav/TEST/neutron-0.71.x/node_modules/react-native-device-info/src/index.ts","original":{"line":38,"column":-1},"name":null}
    at SourceMapGenerator.SourceMapGenerator_validateMapping [as _validateMapping] (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/node_modules/source-map/lib/source-map-generator.js:289:13)
    at SourceMapGenerator.SourceMapGenerator_addMapping [as addMapping] (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/node_modules/source-map/lib/source-map-generator.js:101:12)
    at /Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/src/composeSourceMaps.js:42:15
    at MappingsConsumer.eachMapping (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/src/Consumer/AbstractConsumer.js:34:16)
    at composeSourceMaps (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/src/composeSourceMaps.js:36:16)
    at Object.<anonymous> (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/react-native/scripts/compose-source-maps.js:59:5)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Module._load (node:internal/modules/cjs/loader:827:12)

I could easily to reach the same error just performing create compose map (does not need to build):

--------- 1. Bundle/minify with metro (react-native) to get the packager source map (.packager.map)
+ npx react-native bundle --platform android --dev true --minify false --entry-file index.js --bundle-output 
--------- 2. Compile to bytecode using hermes to get the compiler source map (.hbc.map)
+ ./node_modules/react-native/sdks/hermesc/osx-bin/hermesc -w -O -emit-binary -output-source-map -out=./android/index.android.bundle.hbc ./android/index.android.bundle
--------- 3. Merge the two source maps using compose-source-maps to get the final source map (.map)
+ node node_modules/react-native/scripts/compose-source-maps.js ./android/index.android.bundle.packager.map ./android/index.android.bundle.hbc.map -o ./android/index.android.bundle.map

/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/node_modules/source-map/lib/source-map-generator.js:289
      throw new Error('Invalid mapping: ' + JSON.stringify({
      ^

Error: Invalid mapping: {"generated":{"line":1,"column":1648084},"source":"/Users/alexkrav/TEST/neutron-0.71.x/node_modules/react-native-device-info/src/index.ts","original":{"line":38,"column":-1},"name":null}
    at SourceMapGenerator.SourceMapGenerator_validateMapping [as _validateMapping] (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/node_modules/source-map/lib/source-map-generator.js:289:13)
    at SourceMapGenerator.SourceMapGenerator_addMapping [as addMapping] (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/node_modules/source-map/lib/source-map-generator.js:101:12)
    at /Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/src/composeSourceMaps.js:42:15
    at MappingsConsumer.eachMapping (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/src/Consumer/AbstractConsumer.js:34:16)
    at composeSourceMaps (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/metro-source-map/src/composeSourceMaps.js:36:16)
    at Object.<anonymous> (/Users/alexkrav/TEST/neutron-0.71.x/node_modules/react-native/scripts/compose-source-maps.js:59:5)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Module._load (node:internal/modules/cjs/loader:827:12)
alexkrav commented 1 year ago

package.json:

{
...
  "dependencies": {
    "@evernote/skitch-neutron": "0.17.838",
    "@react-native-community/async-storage": "^1.12.1",
    "@react-native-camera-roll/camera-roll": "^5.2.4",
    "@react-native-community/datetimepicker": "^6.4.2",
    "@react-native-community/google-signin": "^4.0.3",
    "@react-native-community/hooks": "^2.8.0",
    "@react-native-community/image-editor": "^2.2.0",
    "@react-native-community/netinfo": "^9.0.0",
    "@react-native-community/progress-bar-android": "^1.0.3",
    "@react-native-community/progress-view": "^1.1.1",
    "@react-native-community/cookies": "^5.0.1",
    "@react-native-community/slider": "^2.0.7",
    "@react-native-community/viewpager": "^4.2.2",
    "@react-native-firebase/app": "8.1.0",
    "@react-native-firebase/remote-config": "7.2.0",
    "@sentry/react-native": "^4.8.0",
    "@storybook/addon-ondevice-knobs": "^5.1.11",
    "@tradle/react-native-http": "^2.0.1",
    "@types/countries-and-timezones": "^2.0.3",
    "@welldone-software/why-did-you-render": "^4.2.7",
    "appcenter": "4.4.5",
    "appcenter-crashes": "4.4.5",
    "backoff-rxjs": "6.5.5",
    "buffer": "^5.4.2",
    "cheerio": "^1.0.0-rc.5",
    "color": "^3.1.3",
    "conduit-core": "1.65.0",
    "conduit-utils": "1.65.0",
    "conduit-view": "1.65.0",
    "countries-and-timezones": "^2.3.1",
    "crypto-js": "^3.1.9-1",
    "date-fns": "^2.9.0",
    "date-fns-tz": "^1.3.4",
    "debounce-promise": "^3.1.2",
    "en-calendar": "~3.0.186",
    "en-clipper-core": "1.41.0",
    "en-clipper-url-metadata-fetcher": "1.41.0",
    "en-conduit-react-native": "1.65.0",
    "en-conduit-react-native-renderer": "1.65.0",
    "en-conduit-react-native-shared": "1.65.0",
    "en-conduit-schema": "1.65.0",
    "en-experiments": "^3.7.304",
    "en-features-rollout": "^0.1.12",
    "en-react-native-begin-background-task": "1.65.0",
    "en-react-native-local-notifications": "1.65.0",
    "en-react-native-resource-cache": "1.65.0",
    "en-react-native-workers": "1.65.0",
    "en-search-engine-react-native": "^37.0.0",
    "en-task-client-utils": "^2.6.3",
    "en-task-duedate-utils": "^1.35.3",
    "en-task-helpers": "^1.20.3",
    "en-task-schema": "^1.4.1",
    "evernote-ce": "165.2.20482",
    "evernote-ink": "1.0.205",
    "evernote-thrift": "^1.4.4",
    "fast-xml-parser": "^3.19.0",
    "galactic-context": "^3.0.0",
    "graphql": "^16.6.0",
    "graphql-tag": "^2.12.6",
    "he": "^1.2.0",
    "hoist-non-react-statics": "^3.3.0",
    "iconv-lite": "^0.6.2",
    "jsc-android": "^245459.0.0",
    "lodash.merge": "^4.6.2",
    "lottie-ios": "3.4.1",
    "lottie-react-native": "^5.1.4",
    "moment": "^2.29.2",
    "moment-timezone": "^0.5.31",
    "node-libs-react-native": "^1.0.3",
    "plurr": "^1.0.2",
    "postinstall-postinstall": "^2.0.0",
    "pretty-bytes": "^5.3.0",
    "prop-types": "^15.0.0",
    "query-string": "^6.8.3",
    "react": "18.2.0",
    "react-content-loader": "^5.0.4",
    "react-dom": "^18.2.0",
    "react-freeze": "^1.0.0",
    "react-highlight-words": "^0.16.0",
    "react-native": "0.71.3",
    "react-native-actionsheet": "^2.4.2",
    "react-native-activity-view": "^0.2.11",
    "react-native-animation-hooks": "^1.0.1",
    "react-native-autolink": "^4.0.0",
    "react-native-background-timer": "^2.3.1",
    "react-native-camera": "ssh://git@...../neutron/react-native-camera.git#63d1c6e7ed63a415bf1a943e9c723fbada6967ee",
    "react-native-contacts": "^5.0.4",
    "react-native-crash-tester": "^0.2.2",
    "react-native-device-info": "^10.0.2",
    "react-native-document-picker": "^4.0.0",
    "react-native-draggable-flatlist": "^2.2.0",
    "react-native-easy-guesture-responder": "^1.1.3",
    "react-native-geolocation-service": "^5.0.0",
    "react-native-gesture-handler": "^2.9.0",
    "react-native-haptic-feedback": "^1.11.0",
    "react-native-iap": "^12.3.2",
    "react-native-idle-timer": "^2.1.6",
    "react-native-image-crop-picker": "^0.30.0",
    "react-native-image-resizer": "^1.2.2",
    "react-native-image-transformer": "^1.2.1",
    "react-native-inappbrowser-reborn": "^3.7.0",
    "react-native-keyboard-aware-scroll-view": "^0.9.4",
    "react-native-keychain": "4.0.1",
    "react-native-kochava-tracker": "^1.6.0",
    "react-native-linear-gradient": "^2.5.6",
    "react-native-localize": "^2.2.5",
    "react-native-maps": "^1.3.2",
    "react-native-mime-types": "Liroo/react-native-mime-types",
    "react-native-mmkv": "^2.6.2",
    "react-native-modal": "^13.0.0",
    "react-native-modal-translucent": "^5.0.0",
    "react-native-orientation-locker": "^1.1.8",
    "react-native-page-list": "^1.1.2",
    "react-native-pdf": "6.2.2",
    "react-native-permissions": "3.6.1",
    "react-native-popover-view": "^5.0.0",
    "react-native-qualtrics": "2.3.0",
    "react-native-rate": "^1.2.9",
    "react-native-reanimated": "^1.7.0",
    "react-native-redash": "^16.0.8",
    "react-native-render-html": "^6.1.0",
    "react-native-safe-area": "^0.5.1",
    "react-native-safe-area-context": "^4.4.1",
    "react-native-screens": "^1.0.0-alpha.22",
    "react-native-section-list-get-item-layout": "^2.2.3",
    "react-native-securerandom": "^1.0.1",
    "react-native-sfsafariviewcontroller": "^0.2.6",
    "react-native-shadow": "^1.2.2",
    "react-native-shake": "^3.4.0",
    "react-native-splash-screen": "^3.2.0",
    "react-native-sqlite-storage": "4.1.0",
    "react-native-static-safe-area-insets": "^2.1.2",
    "react-native-svg": "^11.0.1",
    "react-native-swipeable": "^0.6.0",
    "react-native-touch-id": "^4.4.1",
    "react-native-url-polyfill": "^1.3.0",
    "react-native-version-number": "^0.3.6",
    "react-native-watch-connectivity": "^0.2.0",
    "react-native-webview": "^11.23.0",
    "react-native-zip-archive": "^6.0.8",
    "react-navigation": "^4.3.9",
    "react-navigation-drawer": "^2.4.13",
    "react-navigation-hooks": "^1.1.0",
    "react-navigation-stack": "^1.9.4",
    "react-navigation-tabs": "^2.5.5",
    "react-xml-parser": "^1.1.6",
    "rn-fetch-blob": "0.13.0-beta.2",
    "rn-placeholder": "^3.0.0",
    "rxjs": "^6.5.3",
    "uuid": "^3.3.2",
    "xmldom": "^0.6.0",
    "xmlbuilder2": "^2.1.2",
    "y-protocols": "^1.0.5",
    "yjs": "^13.5.24"
  },
  "resolutions": {
    "en-thrift-internal": "2.10.1",
    "en-ts-utils": "10.3.2",
    "@babel/core": "^7.20.0",
    "@babel/runtime": "^7.20.0",
    "@evernote/skitch-core": "0.17.838",
    "@types/react": "18.0.24",
    "ua-parser-js": "0.7.28"
  },
  "devDependencies": {
    "@babel/cli": "^7.20.0",
    "@babel/core": "^7.20.0",
    "@babel/plugin-proposal-class-properties": "^7.12.1",
    "@babel/preset-env": "^7.20.0",
    "@babel/runtime": "^7.20.0",
    "@babel/types": "^7.20.0",
    "@evernote/prettierrc": "^1.0.2",
    "@playlyfe/gql": "^2.6.1",
    "@react-native-community/eslint-config": "^3.2.0",
    "@storybook/addon-actions": "^6.1.14",
    "@storybook/addon-knobs": "^6.1.14",
    "@storybook/react-native": "^5.1.11",
    "@storybook/react-native-server": "^5.3.23",
    "@storybook/theming": "^5.1.11",
    "@testing-library/jest-native": "^3.4.3",
    "@testing-library/react-hooks": "3.2.1",
    "@testing-library/react-native": "7.1.0",
    "@tsconfig/react-native": "^2.0.2",
    "@types/browser-bunyan": "^0.4.5",
    "@types/bunyan": "^1.8.6",
    "@types/crypto-js": "^3.1.43",
    "@types/debounce-promise": "^3.1.1",
    "@types/enzyme": "^3.9.0",
    "@types/enzyme-adapter-react-16": "^1.0.5",
    "@types/graphql": "^14.2.0",
    "@types/he": "^1.1.1",
    "@types/hoist-non-react-statics": "^3.3.0",
    "@types/jest": "^29.2.1",
    "@types/jest-when": "^2.7.3",
    "@types/lodash.merge": "^4.6.6",
    "@types/node": "^12.7.8",
    "@types/react": "^18.0.24",
    "@types/react-native": "^0.68.1",
    "@types/react-native-document-picker": "^2.0.1",
    "@types/react-native-draggable-flatlist": "^1.1.1",
    "@types/react-native-safe-area": "^0.5.0",
    "@types/react-test-renderer": "^18.0.0",
    "@types/storybook__addon-knobs": "5.2.1",
    "@types/storybook__react-native": "^3.0.5",
    "@types/uuid": "^3.4.4",
    "@types/xmldom": "0.1.29",
    "@typescript-eslint/eslint-plugin": "4.18.0",
    "@typescript-eslint/parser": "^4.16.1",
    "babel-jest": "^29.2.1",
    "enzyme": "^3.9.0",
    "enzyme-adapter-react-16": "^1.0.5",
    "eslint": "^8.19.0",
    "eslint-config-prettier": "^8.3.0",
    "eslint-plugin-graphql": "^3.1.1",
    "eslint-plugin-import": "^2.23.4",
    "eslint-plugin-import-order-alphabetical": "^1.0.0",
    "eslint-plugin-jsx-a11y": "^6.4.1",
    "eslint-plugin-navigation-hooks": "file:./eslint-plugins/navigation-effects",
    "eslint-plugin-notice": "^0.8.9",
    "eslint-plugin-prettier": "^3.4.0",
    "eslint-plugin-react": "^7.17.0",
    "eslint-plugin-react-hooks": "4.1.2",
    "husky": "^3.0.4",
    "hygen": "4.0.2",
    "jest": "^29.2.1",
    "jest-haste-map": "24.0.0",
    "jest-resolve": "24.1.0",
    "jest-when": "^3.3.1",
    "jetifier": "^1.6.4",
    "json": "9.0.6",
    "lint-staged": "^9.2.3",
    "marked": "^4.0.16",
    "marked-terminal": "^5.1.1",
    "metro-react-native-babel-preset": "^0.73.7",
    "patch-package": "^6.0.7",
    "prettier": "^2.4.1",
    "react-native-mock-render": "^0.1.6",
    "react-native-svg-transformer": "^1.0.0",
    "react-native-typescript-transformer": "^1.2.12",
    "react-test-renderer": "18.2.0",
    "source-map": "^0.7.3",
    "timezone-mock": "1.3.1",
    "ts-jest": "^26.5.0",
    "typescript": "4.8.4",
    "typescript-call-graph": "^0.0.3",
    "wml": "^0.0.83",
    "xcode": "^3.0.1"
  },
  "jest": {
    "preset": "react-native"
  },
  "private": true
}
cortinico commented 1 year ago

Passing over to metro as the error happened during the bundling phase

robhogan commented 1 year ago

Hi @alexkrav - this sounds like it could be https://github.com/facebook/metro/pull/928, related to our switch to Terser as a default minifier. Could you check whether the problem goes away by switching back to the old default - metro-minify-uglify?

// metro.config.js

module.exports = {
  transformer: {
    minifierPath: require.resolve('metro-minify-uglify')
    // ...
  }
  // ...
}

If so, this fix will be part of RN 0.71.4, and you can safely use 'metro-minify-uglify' in the mean time.

alexkrav commented 1 year ago

Hi @robhogan , Thank you for suggestion. I've added line proposed into metro.config.js../transformer but unfortunately, it did not fix the problem - the same error reported.

alexkrav commented 1 year ago

@robhogan , I've tried to use (forced to resolved to) newest metro-source-map v.0.75.1 with your fix for #928 and run the following the following script:

--------- 1. Bundle/minify with metro (react-native) to get the packager source map (.packager.map)
+ npx react-native bundle --platform android --dev false --minify false --entry-file index.js --bundle-output 
--------- 2. Compile to bytecode using hermes to get the compiler source map (.hbc.map)
+ ./node_modules/react-native/sdks/hermesc/osx-bin/hermesc -w -O -emit-binary -output-source-map -out=./android/index.android.bundle.hbc ./android/index.android.bundle
--------- 3. Merge the two source maps using compose-source-maps to get the final source map (.map)
+ node node_modules/react-native/scripts/compose-source-maps.js ./android/index.android.bundle.packager.map ./android/index.android.bundle.hbc.map -o ./android/index.android.bundle.map

still fails with the same mapping error. Note that bundle created with --minify false param.

robhogan commented 1 year ago

Thanks for trying that - definitely seems like a different issue and one we're unaware of.

Would you be able to put up a minimal GitHub repo or steps I can use to reproduce from a new project? I'll have a bit of time tomorrow to try to investigate if so.

alexkrav commented 1 year ago

@robhogan This is very big project and I do not know how to extract reproducible part.

What I've found - that if bundle is created with --minify true then above script finishes successfully.

robhogan commented 1 year ago

I'd like to get to the bottom of this but I'm stuck without a reproduction. On a new project this works:

$ npx react-native@latest init RNMetroIssue938 && cd RNMetroIssue938
$ node node_modules/react-native/scripts/compose-source-maps.js ./android/index.android.bundle.packager.map ./android/index.android.bundle.hbc.map -o ./android/index.android.bundle.map
$ ./node_modules/react-native/sdks/hermesc/osx-bin/hermesc -w -O -emit-binary -output-source-map -out=./android/index.android.bundle.hbc ./android/index.android.bundle
$ npx react-native bundle --platform android --dev false --minify false --entry-file index.js --bundle-output ./android/index.android.bundle --sourcemap-output ./android/index.android.bundle.packager.map

So, the issue seems to be triggered by something in your project code or the dependencies it imports.

If you have time, I'd suggest taking a clone of your project (which I'm guessing you can't share?) and stub / comment / delete big chunks of it until the problem goes away, then work back from there to narrow it down to maybe a bit of code or particular dependency, and then try to reproduce it with just that bit in a new app.

Hless commented 1 year ago

Chiming in on this issue, not sure if related.

I'm on RN 0.66.5 still, recently upgraded from 0.66.4, and this exact issue started happening n my project as well. Bundling with --minify true also fixes it for me. I'm at a complete loss how this suddenly started happening.

I'm guessing it's a transient dependency that got updated in the meantime that is causing this. We have a massive amount of deps in a monorepo, so it would be extremely difficult to track the culprit down.

@alexkrav Do you have any progress on this issue so far?

alexkrav commented 1 year ago

@Hless No progress with this. Using --minify true On 0.71: to avoid build crash I had to uncomment/modify two options in app/build.gradle:

...
    /* Variants */
    //   The list of variants to that are debuggable. For those we're going to
    //   skip the bundling of the JS bundle and the assets. By default is just 'debug'.
    //   If you add flavors like lite, prod, etc. you'll have to list your debuggableVariants.
    debuggableVariants = ["regularDebug"]
...
    /* Bundling */
    extraPackagerArgs = [ '--minify=true', '--dev=false' ]
xilin commented 1 year ago

In our case the error msg is as follow, failed to find original info?

~/test/node_modules/metro-source-map/node_modules/source-map/lib/source-map-generator.js:289
      throw new Error('Invalid mapping: ' + JSON.stringify({
      ^

Error: Invalid mapping: {"generated":{"line":1,"column":264},"source":"~/test/node_modules/metro-runtime/src/polyfills/require.js","original":{"line":0,"column":0},"name":null}