search

containerbuildsystem / cachi2

Cachi2 is a CLI tool that pre-fetches your project's dependencies to aid in making your build process network-isolated.
GNU General Public License v3.0
7 stars 25 forks source link

Not fetching all NPM dependencies #595

Closed zregvart closed 1 month ago

zregvart commented 1 month ago

Reported on behalf of @kahboom; cachi2 doesn't seem to fetch all dependencies for this repository:

https://github.com/kahboom/rekor-search-ui at 144e299c83f79441dc4b87c078d7d436857f796f. To reproduce run:

$ git clone https://github.com/kahboom/rekor-search-ui.git
$ cd rekor-search-ui
$ git checkout 144e299c83f79441dc4b87c078d7d436857f796f
$ alias cachi2='podman run --rm -ti -v "$PWD:$PWD:z" -w "$PWD" quay.io/redhat-appstudio/cachi2:latest'
$ cachi2 fetch-deps \
  --source . \
  --output ./cachi2-output \
  '{"type": "npm", "path":"."}'
2024-08-12 13:52:10,477 INFO Verifying checksums of next-swc-darwin-x64-13.5.6.tgz
2024-08-12 13:52:10,556 INFO Verifying checksums of next-swc-linux-arm64-gnu-13.5.6.tgz
2024-08-12 13:52:10,632 INFO Verifying checksums of next-swc-linux-arm64-musl-13.5.6.tgz
2024-08-12 13:52:10,713 INFO Verifying checksums of next-swc-linux-x64-gnu-13.5.6.tgz
2024-08-12 13:52:10,785 INFO Verifying checksums of next-swc-linux-x64-musl-13.5.6.tgz
2024-08-12 13:52:10,873 INFO Verifying checksums of next-swc-win32-arm64-msvc-13.5.6.tgz
2024-08-12 13:52:10,932 INFO Verifying checksums of next-swc-win32-ia32-msvc-13.5.6.tgz
2024-08-12 13:52:10,982 INFO Verifying checksums of next-swc-win32-x64-msvc-13.5.6.tgz
2024-08-12 13:52:11,419 INFO All dependencies fetched successfully \o/
$ npm why yocto-queue # note npm why doesn't find yocto-queue
npm error No dependencies found matching yocto-queue

Workaround seems to be running npm dedup:

$ npm dedup
added 1795 packages, and audited 1796 packages in 2m

312 packages are looking for funding
  run `npm fund` for details

13 vulnerabilities (6 low, 7 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
$ cachi2 fetch-deps \
  --source . \
  --output ./cachi2-output \
  '{"type": "npm", "path":"."}'
2024-08-12 13:48:50,742 INFO Verifying checksums of jridgewell-gen-mapping-0.3.5.tgz
...
2024-08-12 13:48:51,413 INFO Verifying checksums of color-name-1.1.4.tgz
2024-08-12 13:48:51,845 INFO All dependencies fetched successfully \o/
$ npm why yocto-queue # npm now finds yocto-queue
yocto-queue@1.1.1 dev
node_modules/babel-loader/node_modules/yocto-queue
  yocto-queue@"^1.0.0" from p-limit@4.0.0
  node_modules/babel-loader/node_modules/p-limit
...
taylormadore commented 1 month ago

The package-lock.json file looks malformed to me. It's missing resolved and integrity keys for nearly all of the dependencies

zregvart commented 1 month ago

When I ran npm install the package-lock.json file was left unchanged, it seems that NPM thinks the lock file is okay.

taylormadore commented 1 month ago

The first thing I tried was deleting package-lock.json and regenerating it by running npm install. All of the missing keys were added back.

It might be related to this issue: https://github.com/npm/cli/issues/4263

In any case, cachi2 does not execute npm itself when prefetching dependencies, so the resolved keys in package-lock.json are required in order for cachi2 to know what to download.

zregvart commented 1 month ago

@taylormadore thanks for the effort and explanation, I think we can close this now -- regenerating the lock file seems to be the way to go.