search

facebook / metro

🚇 The JavaScript bundler for React Native
https://metrobundler.dev
MIT License
5.22k stars 626 forks source link

`SHA-1 for file ... is not computed` when using `resolver.resolveRequest` #330

Open vjpr opened 5 years ago

vjpr commented 5 years ago

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

I am using a custom resolver.resolveRequest function in rn-cli.config.js. See https://facebook.github.io/metro/docs/en/configuration#resolver-options.

The error message is being printed from metro/src/node-haste/DependencyGraph.js#getSha1, which is being called by metro/src/Bundler.js#transformFile:

metro/src/node-haste/DependencyGraph.js#getSha1

  getSha1(filename: string): string {
    const resolvedPath = fs.realpathSync(filename);
    const sha1 = this._hasteFS.getSha1(resolvedPath);
    if (!sha1) {
      throw new ReferenceError(`SHA-1 for file ${filename} is not computed`);
    }
    return sha1;
  }

The issue is that the file won't exist in the haste map if you are resolving it in the resolveRequest hook.

If the current behavior is a bug, please provide the steps to reproduce and a minimal repository on GitHub that we can yarn install and yarn test.

https://github.com/vjpr/expo-v2-test-pnpm/tree/broken

You need to run pnpm install (the reason for the custom resolver is because symlinks are not supported by metro and hence cause problems for pnpm).

What is the expected behavior?

getSha1 should compute hash if not in haste map.

Please provide your exact Metro configuration and mention your Metro, node, yarn/npm version and operating system.

metro@0.45.6 node@8.9.0 pnpm@2.21.1

Workaround

Add to metro/src/node-haste/DependencyGraph.js#getSha1:

    if (!sha1) {
      return getFileHash(resolvedPath)
      function getFileHash(file) {
        return require('crypto')
          .createHash('sha1')
          .update(fs.readFileSync(file))
          .digest('hex')
      }
    }

I have released a patched version here: Use metro-pnpm@0.45.6-vjpr.2

Stephen2 commented 5 years ago

I got this when trying to get Expo to work with yarn workspaces, using expo-yarn-workspaces

pavermakov commented 5 years ago

I've got the same issue when I tried to upgrade react-native to 0.57.0. I followed the steps mentioned the change log (https://github.com/react-native-community/react-native-releases/blob/master/CHANGELOG.md#057), but had no luck.

mareksuscak commented 5 years ago

@Stephen2 me too, did you solve it?

Stephen2 commented 5 years ago

Sorry, moved to different project

ekmobile commented 5 years ago

This is still a problem today with "react-native": "^0.59.2". Will there be real fix or do we need to copy/paste the workaround?

nick-michael commented 5 years ago

For anyone seeing this error when using expo-yarn-workspaces, I got this and fixed it by simply deleting the auto-generated file (the file that the error points to) and deleting all the node modules and then just running yarn again to re-install node modules and re-generate the file.

ghost commented 5 years ago

I am experiencing this on RN 0.60.5

dioptre commented 5 years ago

Also fixes RN 0.61

ghost commented 5 years ago

Also fixes RN 0.61

Do you mean that on RN 0.61 it is working as expected?

donholly commented 5 years ago

Seeing the same issue as described on 0.60.5. The workaround provided by @vjpr works - thank you! Looks like #455 should fix this. I'm curious why doesn't seem to be a bigger problem for everyone 😄

I made the change in ~/.config/yarn/global/node_modules/metro/src/node-haste/DependencyGraph.js

dioptre commented 5 years ago

@dusan-dragon I needed the @vjpr fix to get RN 0.61 to work.

ricick commented 4 years ago

I'm having this issue on a windows 10 machine here. Same project on another dev machine with the same version of node doesn't display the issue.

ikbal99 commented 4 years ago

put all files inside assets name of folder something like this

../../assets/beach.jpg

vj-raghavan commented 4 years ago

@donholly I am not sure why no one is flagging this issue more. I have just run in to this issue after I have setup a new terminal and cleaned up all my old node modules.

Thanks to @vjpr workaround it worked after rebuilding the app. Is this a miss on our part ? or is this being actively looked into ?

mhers91 commented 4 years ago

put all files inside assets name of folder something like this

../../assets/beach.jpg

This worked for me ikbal99

Gerlison commented 4 years ago

Works for me, 0.61.5

nkraev commented 4 years ago

0.61.5 indeed works. I also did npm i -g react-native-cli - it updated cli to 2.0.1 version and bundling worked.

bkhurjawal commented 4 years ago

I solved this issue by using yarn startinstead of starting project withreact-native start(it uses npm start). I have been adding dependencies in project using Yarn add {dependencyName} and starting project via npm. which created error of SHA-1.

ricick commented 4 years ago

I found it was an issue with me accessing the project on a second drive via a symlink on C: rather than navigating to the D: drive

f4th4n commented 4 years ago

I fixed it by install correct react-native-cli:

yarn global remove react-native
yarn global add react-native-cli
martinfrouin commented 4 years ago

Thanks @f4th4n it works for me

finghtingjie commented 4 years ago

I had tried all the methods above but still not work for me. OS:mac Catalina node:13.3.0 watchman:4.9.0 react-native:0.61.5

the error log like this

Metro Bundler has encountered an error: SHA-1 for file /Users/.../Downloads/AwesomeProject/node_modules/metro/src/lib/polyfills/require.js (/Users/.../Downloads/AwesomeProject/node_modules/metro/src/lib/polyfills/require.js) is not computed

MohamedHamedLab commented 4 years ago

i had the same issue and fix it by following this steps 1- updating metro-react-native-babel-preset to version 0.57.0
2- resetCache: true in metro.config 3- removeing node modules & yarn lock file and reinstall

finghtingjie commented 4 years ago

i had the same issue and fix it by following this steps 1- updating metro-react-native-babel-preset to version 0.57.0 2- resetCache: true in metro.config 3- removeing node modules & yarn lock file and reinstall

not work for me

alancwoo commented 4 years ago

I just ran into this issue while enjoying the fun task of upgrading from 0.59.1 to 0.61.5 with react-native-cli.

My error made me realize I was running the bundler via a copy of react-native that was installed to nvm (Node Version Manager). Running which react-native confirmed this: /Users/XXXXXX/.nvm/versions/node/v10.16.2/bin/react-native

I made sure to remove all 'global' installs of react-native:

  1. npm -g remove react-native
  2. yarn global remove react-native
  3. brew uninstall react-native-cli

Until which react-native responded with react-native not found

Then in my project, npx react-native run-ios worked.

--

Mac OS X 10.14.6 RN 0.61.5 npx react-native --version 3.0.4

tourze commented 4 years ago

same problem

ymotse commented 4 years ago

I fixed it by install correct react-native-cli:

yarn global remove react-native
yarn global add react-native-cli

Works for me!!! :D

yashvanzara commented 4 years ago

I fixed it by install correct react-native-cli:

yarn global remove react-native
yarn global add react-native-cli

npm start worked for me

alincbiz commented 4 years ago

I made an update to jdk, version 11, and it works for me

acmedent commented 4 years ago

I have installed again the react-native-cli and worked!

smondal229 commented 4 years ago

0.61.5 indeed works. I also did npm i -g react-native-cli - it updated cli to 2.0.1 version and bundling worked.

currently my project is on 0.61.5 but it showing the same error

leocavalcante commented 4 years ago

Just got this with "react-native": "0.61.5", I installed react-native-cli and initialized a new project today 😛

sayzenberg commented 4 years ago

+1, ran into this with react-native 0.61.5 and react-native-cli 2.0.1. The only thing that's worked for me is the workaround provided by @vjpr.

jhalvorson commented 4 years ago

I've just ran into this with react-native: 0.61.2. @vjpr's workaround has temporarily fixed it for me.

I've tried lots of things including:

However, the above workaround is the only thing that's fixed it for me

velsa commented 4 years ago

Ran into the same issue.

In my case, it complained about not being able to create SHA-1 for timezones.json, which is a package in my node_modules (https://www.npmjs.com/package/timezones.json). Since this package was a dependency of another package, my solution was to rename the directory timezones.json to timezones-json and also change the require in the corresponding package.

It seems that metro looks at the extension of the file and not at the file type, which caused it to think that a directory was a .json file.

a7urag commented 4 years ago

@vjpr solution worked for me, but any permanent solution for this?

clayrisser commented 4 years ago

@vjpr solution worked for me also. Is there a reason that code is not merged in?

leandrosouzaa commented 4 years ago

I solved this issue by using yarn startinstead of starting project withreact-native start(it uses npm start). I have been adding dependencies in project using Yarn add {dependencyName} and starting project via npm. which created error of SHA-1.

@bkhurjawal worked for me, thanks!!

clayrisser commented 4 years ago

I wrote a patch for it that works with the existing metro package.

  1. Install the following dependency

    npm install --save mod-inline

  2. Create the following file

    install.js

    const fs = require('fs');
const modInline = require('mod-inline').default;
const path = require('path');

const GET_FILE_HASH = / {4}if \(!sha1\) {\n {6}return getFileHash\(resolvedPath\);/;
const GET_SHA1_FUNC = / {2}getSha1\(filename\) {((\n {4}.*|\n)+)+ {2}}/;
const IF_SHA1 = / {4}if \(!sha1\) {/;

const dependencyGraphPath = path.resolve(
  __dirname,
  '../node_modules/metro/src/node-haste/DependencyGraph.js'
);
const content = fs.readFileSync(dependencyGraphPath).toString();
const patched = !!modInline.isolate(
  modInline.isolate(content, GET_SHA1_FUNC),
  GET_FILE_HASH
);
if (!patched) {
  const patchedContent = modInline.append(
    content,
    [GET_SHA1_FUNC, IF_SHA1],
    `
      return getFileHash(resolvedPath);
      function getFileHash(file) {
        return require('crypto')
          .createHash('sha1')
          .update(fs.readFileSync(file))
          .digest('hex');
      }`
  );
  fs.writeFileSync(dependencyGraphPath, patchedContent);
}

  3. Add the following script to the package.json

{
  "scripts": {
    "install": "node ./install.js"
  }
}
alexyazvinsky commented 4 years ago

same issue, tried on several osx and only @vjpr's solution helped any other workarounds? and why is it not fixed officially?

OS:mac Catalina 10.15.4 node:10.16.3 watchman:4.9.0 react-native:0.62.2

a7urag commented 4 years ago

Finally got what went wrong here, folder name has 1 upper case character in require. eg. require(../../../assets/marketPlace/banner.png). Here "p" was upper case here but folder was with lower case p. After changing require path it was working again.

coolcorexix commented 4 years ago

Hello, I have this problem and then fixed it by using directly the node_modules/react-native/scripts/launchPackager.command instead of react-native start

If you have any further investigation on this please let me know

SeanHayes commented 4 years ago

I have this same error with the following code:

import { createIconSet } from '@expo/vector-icons';

const glyphMap = {
    'kk-logo': 0xf101,
};
const expoAssetId = require("../.cache/iconfont/iconfont.ttf");
const CustomIcon = createIconSet(glyphMap, 'iconfont', expoAssetId);

export default CustomIcon;

../.cache/iconfont/iconfont.ttf is a custom font generated using svgs.

It was working fine, then my Macbook restarted after an update, now I get this error no matter what I do. Tried clearing caches, tmp directories, etc. but nothing works.

samuelneff commented 4 years ago

I created a new blank React Native app and got this same error about SHA1 and require.js. My React Native application name had a mixed-case name, ex. MyApp. Recreated the same app a half dozen times with different versions of node and React Native and same error each time.

Then I created the same app but named it myapp and didn't have this problem. It just worked.

dannycoulombe commented 4 years ago

Thanks @vjpr, you saved me hours. I can't believe Github doesn't offer some sort of donation button.

vjpr commented 4 years ago

Just an update on why this happens.

Here is the function that is generating the message:

https://github.com/facebook/metro/blob/master/packages/metro/src/node-haste/DependencyGraph.js#L208

  getSha1(filename) {
    // TODO If it looks like we're trying to get the sha1 from a file located
    // within a Zip archive, then we instead compute the sha1 for what looks
    // like the Zip archive itself.
    const splitIndex = filename.indexOf(".zip/");
    const containerName =
      splitIndex !== -1 ? filename.slice(0, splitIndex + 4) : filename; // TODO Calling realpath allows us to get a hash for a given path even when
    // it's a symlink to a file, which prevents Metro from crashing in such a
    // case. However, it doesn't allow Metro to track changes to the target file
    // of the symlink. We should fix this by implementing a symlink map into
    // Metro (or maybe by implementing those "extra transformation sources" we've
    // been talking about for stuff like CSS or WASM).

    const resolvedPath = fs.realpathSync(containerName);

    const sha1 = this._hasteFS.getSha1(resolvedPath);

    if (!sha1) {
      throw new ReferenceError(
        `SHA-1 for file ${filename} (${resolvedPath}) is not computed`
      );
    }

    return sha1;
  }

Here is the function that gets the sha1 from the haste map:

jest-haste-map/build/worker.js

function _getSha() {
  _getSha = _asyncToGenerator(function*(data) {
    const sha1 = data.computeSha1
      ? sha1hex(_gracefulFs().default.readFileSync(data.filePath))
      : null;
    return {
      dependencies: undefined,
      id: undefined,
      module: undefined,
      sha1
    };
  });
  return _getSha.apply(this, arguments);
}

When metro resolves a file it looks for the file in a jest-haste-map which also watches for changes to these files. There are two implementations for building a haste map (crawling the file tree from the project roots):

  1. Facebook's watchman native (c++, rust, etc.) library. jest-haste-map/build/crawlers/watchman.js
  2. Node.js implementation jest-haste-map/build/crawlers/node.js

Both don't support following symlinks...so if you have any symlinks in your project root that point to files outside of the project root, then they will never make it into the haste map which also means they will never be watched. It was the first feature request for metro 5 years ago and its unresolved. You can see the issue here: https://github.com/facebook/metro/issues/1#issuecomment-641633646

So there are two situations here.

a. Your symlink points to a file that is inside your project root.

b. Your symlink points to a file outside of your project root. E.g. If you are using a pnpm-based monorepo, all your symlinks resolve to the monorepo root. So if expo is nested inside the monorepo, then you symlinks will always point outside the project root.

For a, your sha-1 will actually be found, because your file's realpath will be in the haste map. For b, your sha-1 will not be found because your file's realpath is not in the haste map.

When metro requires a file it needs to transform it. It first checks if its transformation is cached in a metro-cache. The sha-1 of the file is used as part of the cache key.

If its not in the cache, the file contents is actually read not from the haste map but from the file system. So if your file was symlinked to outside of a project root, it won't be in the haste map at all, and the sha1 can't be accessed...but it will still be able to be loaded fine.

https://github.com/facebook/metro/blob/7814c2840c49c16788041284fd65df25aa997d8c/packages/metro/src/DeltaBundler/Transformer.js#L141-L147

Re: extraNodeModules

If you look inside metro-resolver#resolve you can see that extraNodeModules[packageName] is added to the list of search paths.

https://github.com/facebook/metro/blob/7814c2840c49c16788041284fd65df25aa997d8c/packages/metro-resolver/src/resolve.js#L108-L134

allDirPaths looks like this:

"/dev/my-monorepo/packages/app-templates/expo/node_modules/@babel/runtime/helpers/interopRequireDefault"
"/dev/my-monorepo/packages/app-templates/node_modules/@babel/runtime/helpers/interopRequireDefault"
"/dev/my-monorepo/packages/node_modules/@babel/runtime/helpers/interopRequireDefault"
"/dev/my-monorepo/node_modules/@babel/runtime/helpers/interopRequireDefault"
"/dev/node_modules/@babel/runtime/helpers/interopRequireDefault"
"/node_modules/@babel/runtime/helpers/interopRequireDefault"
"/dev/my-monorepo/packages/app-templates/expo/node_modules/@babel/runtime/helpers/interopRequireDefault"

Note how the last path comes from the extraPaths.

NOTE: These paths are candidate paths, not dirs to search for packages in.

The extraNodeModules proxy hack is clever, as it allows you to take the packageName and resolve it how you would like. Similar could be done with resolveRequest but its more complicated.

A downside though is you do not get access to where the require is being made from...so it is only really viable if your package is hoisted in yarn. This is unreliable though! If you have two packages of same version, then only one is hoisted and the other is not which could cause crazy bugs.

Overriding resolveRequest is the much safer option.

One spanner in the works though is that inside the resolver context.doesFileExist is called to check if the logical file path exists - not the realpath.

In pnpm:

$ realpath /dev/my-monorepo/packages/expo/node_modules/@babel/runtime/helpers/interopRequireDefault.js

/dev/my-monorepo/node_modules/.pnpm/registry.npmjs.org/@babel/runtime/7.10.2/node_modules/@babel/runtime/helpers/interopRequireDefault.js

So for a monorepo in a pnpm setup, this will prevent files being found, even if you add the monorepo root to the watchFolders.

https://github.com/facebook/metro/blob/7814c2840c49c16788041284fd65df25aa997d8c/packages/metro-resolver/src/resolve.js#L423-L433

  _getFileData(file) {
    const relativePath = fastPath.relative(this._rootDir, file);
    return this._files.get(relativePath);
  }

Including the entirety of the monorepo as a watchFolder may slow down the app startup. Expo will actually work without needing to include every node_module, but then you need to make the sha1 patch and doesFileExist patch.

We can modify ctx.doesFileExist in the resolver#resolveRequest metro config setting. I'm not sure if this will cause problems if resolve a logical path rather than a real path.

LATER: Maybe we could use hasteImplModulePath option to provide a custom haste map that supports symlinks too.

In terms of a clean solution, I think it is best to ensure that all the files that you would possibly require make their way into the haste map, which I think is possible by setting watchFolders in the metro config.

Summary

I'm leaving this long explanation here because I had completely forgotten it since I last looked at it, and was hoping there was a fix by now but it seems not.

  1. Try to include everything in the haste map.

Symlinks will not be followed, so you will need to make sure the folders they point to are added to the haste map by including them as watchFolders.

extraNodeModules doesn't add files to the haste map, it will only provide additional candidate resolve paths...but they will still need to be in the haste map.

  1. If you can't put everything in the haste map, then you will need to patch the getSha1 function, and you might need to patch the metro-resolve#resolve function.

I'm thinking there might be a way to monkey-patch the library without needing a patch.

Just need to intercept the require of node_modules/metro/src/node-haste/DependencyGraph.js and override its getSha1 method. Could probably add the require hook when metro.config.js is called...

vjpr commented 4 years ago

I think I found a simple explanation for this problem. See https://github.com/facebook/jest/issues/10063#issuecomment-642356808.

When metro talks to watchman it sends a query with a glob option. When a glob is used it prevents files and folders starting with a dot from being returned by the watchman query. There is an option to pass called glob_includedotfiles which includes dot files.

Depending on whether the clock is available, if glob is not used, then you are not going to get any dotfiles or dot dirs included.

So this explains the sporadic behaviour.

const relativeRoot = fastPath.relative(rootDir, root);
const query = clocks.has(relativeRoot) // Use the `since` generator if we have a clock available
  ? {
      expression,
      fields,
      since: clocks.get(relativeRoot)
    } // Otherwise use the `glob` filter
  : {
      expression,
      fields,
      glob
    };
const response = yield cmd('query', root, query);

This explains why @SeanHayes was having a problem until his MacBook restarted.

And it also explains why pnpm doesn't work because all its deps are found in node_modules/.pnpm which includes a dir starting with a dot.

Its a one line fix that is in the pipeline but I feel like I went to the ends of the earth to track it down.

wannymiarelli commented 4 years ago

@vjpr

You are right about symlinks, on windows I had this problem with the "Documents" directory. It is spelt "Documenti" in Italian if you run the "yarn start" using the "Documenti" alias instead of the "Documents" it fails with the SHA-1 computed error.

So c:/users/[username]/Documenti - (Italian version) fails c:/users/[username]/Documents - works

Running the command from the "Documents" path works just fine.

jinyuyoulong commented 4 years ago

I make it work for this: npx react-native bundle --platform ... but not work when I use like this: react-native bundle --platform

didn't know why!

zilurrane commented 4 years ago

Same experience like @jinyuyoulong