search

jfrog / jfrog-cli

JFrog CLI is a client that provides a simple interface that automates access to the JFrog products.
https://www.jfrog.com/confluence/display/CLI/JFrog+CLI
Apache License 2.0
536 stars 235 forks source link

Unable to gather build info for Node app w/ `npm` #2702

Open eric-gonzalez-tfs opened 1 month ago

eric-gonzalez-tfs commented 1 month ago

Describe the bug

The JFrog CLI is unable to publish build info for my application. I see errors regarding failed checksum calculations.

Current behavior

This shows the initial setup that I have been using to test:

Screenshot 2024-09-20 at 5 19 39 PM

Moments later the app runs it's npm preinstall and prepare scripts, and we start seeing some additional details populate the logs...

and it goes on forever......

- Finally this warning regarding not dependencies not being captured in build-info:
```bash
17:28:35 [🟠Warn] The following dependencies will not be included in the build-info, because they are missing in the npm cache: '@babel/helper-hoist-variables:7.22.5,reusify:1.0.4,esquery:1.5.0,jsonparse:1.3.1,destroy:1.2.0,micromark-util-subtokenize:2.0.1,file-type:19.0.0,is-plain-obj:2.1.0,clone:1.0.4,argparse:1.0.10,basic-ftp:5.0.5,moment:2.30.1,parse-path:7.0.0,normalize-url:8.0.1,oas-validator:5.0.8,require-directory:2.1.1,yoctocolors-cjs:2.1.2,minipass:5.0.0,agent-base:6.0.2,run-async:2.4.1,micromark-factory-label:2.0.0,cosmiconfig:8.2.0,open-editor:4.1.1,iconv-lite:0.4.24,bl:4.1.0,to-regex-range:5.0.1,send:0.18.0,boxen:7.1.1,@webassemblyjs/wast-printer:1.12.1,@babel/helper-module-transforms:7.24.5,emoji-regex:10.3.0,micromark-factory-whitespace:1.1.0,yargs:17.7.2,wcwidth:1.0.1,yaml:2.3.4,signal-exit:3.0.7,ansi-regex:6.0.1,is-glob:2.0.1,semver-diff:4.0.0,remark-lint-strong-marker:3.1.2,@webassemblyjs/wasm-opt:1.12.1,eslint-plugin-n:16.6.2,json-stable-stringify:1.1.1,@types/mocha:10.0.6,content-type:1.0.5,unist-util-remove-position:5.0.0,longest-streak:3.1.0,eslint-visitor-keys:2.1.0,openapi-validator-middleware:2.0.1,@webassemblyjs/helper-api-error:1.11.6,minimatch:9.0.5,@types/swagger-ui-express:4.1.6,async-listener:0.6.10,workerpool:6.2.1,kind-of:6.0.3,qrcode:1.5.3,@sindresorhus/is:5.6.0,ansi-align:3.0.1,remark-lint-definition-case:3.1.2,remark-lint-no-table-indentation:4.1.2,meow:13.2.0,@types/wait-on:5.3.4,parseurl:1.3.3,unified-lint-rule:2.1.2,glob-to-regexp:0.4.1,is-docker:2.2.1,mkdirp:3.0.1,memorystream:0.3.1,spdx-exceptions:2.5.0,mute-stream:0.0.8,log-symbols:4.1.0,graceful-fs:4.2.11,sort-package-json:2.10.0,wildcard-match:5.1.3,@babel/helper-string-parser:7.24.1,deep-is:0.1.4,cacache:18.0.4,path-is-absolute:1.0.1,is-stream:2.0.1,lint-staged:15.2.2,available-typed-arrays:1.0.7,remark-lint-final-newline:2.1.2,@commitlint/is-ignored:19.2.2,http-status-codes:2.3.0,http2-wrapper:2.2.1,oas-linter:3.2.2,jsesc:3.0.2,url-or-path:2.3.0,type-fest:0.21.3,append-field:1.0.0,chalk:5.3.0,tar-fs:2.1.1,brace-expansion:1.1.11,indent-string:4.0.0,remark-lint-code-block-style:3.1.2,@eslint/js:8.57.0,@babel/parser:7.23.0,progress:2.0.3,http-errors:2.0.0,color-name:1.1.4,cli-spinners:2.9.2,dir-glob:3.0.1,array.prototype.flat:1.3.2,find-cache-dir:5.0.0,mkdirp-classic:0.5.3,tar-stream:3.1.7,call-bind:1.0.7,json-schema-ref-parser:6.1.0,joi:17.13.1,p-limit:3.1.0,optionator:0.9.4,lodash.upperfirst:4.3.1,@octokit/auth-token:4.0.0,@typescript-eslint/typescript-estree:7.10.0,is-get-set-prop:1.0.0,esm-utils:4.3.0,@nicolo-ribaudo/eslint-scope-5-internals:5.1.1-v1,type-fest:4.21.0,bare-events:2.4.2,core-util-is:1.0.3,continuation-local-storage:3.2.1,typed-array-byte-offset:1.0.2,mdast-util-mdx-expression:1.3.2,lodash.startcase:4.4.0,bare-path:2.1.3,unique-slug:4.0.0,minipass:3.3.6,postgres-range:1.1.4,@npmcli/config:8.3.2,@types/acorn:4.0.6,valid-url:1.0.9,mimic-response:3.1.0,camelcase:7.0.1,mdast-util-gfm-autolink-literal:2.0.0,mdast-util-phrasing:3.0.1,@commitlint/top-level:19.0.0,bare-fs:2.3.1,globby:11.1.0,cpe-fs:1.0.1,read-pkg:3.0.0,is-typedarray:1.0.0,micromark-extension-gfm-autolink-literal:2.0.0,array.prototype.findlastindex:1.2.5,@types/json5:0.0.29,@babel/compat-data:7.24.4,ini:4.1.1,readable-stream:3.6.2,conventional-changelog-codemirror:4.0.0,tar-fs:3.0.6,p-is-promise:3.0.0,accepts:1.3.8,globby:14.0.2,external-editor:3.1.0,properties-reader:2.3.0,inquirer:8.2.5,shebang-command:2.0.0,etag:1.8.1,is-regex:1.1.4,@octokit/rest:20.1.1,socks:2.8.3,unicode-emoji-modifier-base:1.0.0,deep-extend:0.6.0,process-nextick-args:2.0.1,conventional-commit-types:3.0.0,is-ci:3.0.1,path-scurry:1.11.1,jsonpath-plus:7.2.0,typed-array-length:1.0.6,lru-queue:0.1.0,proto-list:1.2.4,eslint-compat-utils:0.5.0,is-js-type:2.0.0,@types/ssh2-streams:0.1.12,isexe:2.0.0,copyfiles:2.4.1,which-boxed-primitive:1.0.2,rechoir:0.6.2,remark-lint-emphasis-marker:3.1.2,lodash.snakecase:4.1.1,webidl-conversions:3.0.1,data-view-buffer:1.0.1,postgres-array:2.0.0,require-main-filename:2.0.0,events:3.3.0,common-path-prefix:3.0.0,@babel/helpers:7.24.5,delay:6.0.0,diagnostic-channel:0.3.1,p-try:2.2.0,unist-util-position:4.0.4,to-absolute-glob:3.0.0,callsites:3.1.0,cacheable-request:10.2.14,micromark-factory-whitespace:2.0.0,estraverse:4.3.0,flat-cache:3.2.0,split2:4.2.0,debug:4.3.5,which:4.0.0,shebang-regex:3.0.0,eventemitter3:5.0.1,wrap-ansi:7.0.0,abbrev:2.0.0,@types/ms:0.7.34,untildify:4.0.0,unist-util-stringify-position:4.0.0,is-arrayish:0.2.1,cpu-features:0.0.10,@nodelib/fs.scandir:2.1.5,lazystream:1.0.1,http-cache-semantics:4.1.1,@mycompany/my-package:6.1.3,@npmcli/name-from-folder:2.0.0,@types/mdast:3.0.15,mdast-util-mdx-jsx:3.1.2,peek-readable:5.0.0,fast-diff:1.3.0,@eslint-community/regexpp:4.10.0,socks-proxy-agent:8.0.4,is-ssh:1.4.0,didyoumean:1.2.2,get-intrinsic:1.2.4,eslint-mdx:3.1.5,sade:1.8.1,encodeurl:1.0.2,forwarded:0.2.0,concat-stream:1.6.2,openapi-schema-validation:0.4.2,@xtuc/ieee754:1.2.0,import-modules:2.1.0,debug:4.3.4,bindings:1.5.0,ajv-formats:2.1.1,listr2:8.0.1,object-assign:4.1.1,pg-types:4.0.2,has-flag:3.0.0,p-cancelable:3.0.0,json5:2.2.3,dargs:8.1.0,buffer-crc32:0.2.13,tr46:0.0.3,es-errors:1.3.0,resolve-dir:1.0.1,log-update:6.0.0,pidtree:0.6.0,mocha:10.4.0,node-pty:1.0.0,memoizee:0.4.15,flatted:3.3.1,@commitlint/config-validator:19.0.3,zwitch:2.0.4,ansi-colors:4.1.1,@types/http-cache-semantics:4.0.4,micromark-extension-gfm-task-list-item:2.0.1,@babel/helper-function-name:7.23.0,fast-deep-equal:3.1.3,locate-path:7.2.0,dot-prop:5.3.0,json-schema-ref-parser:7.1.4,get-stream:5.2.0,y18n:4.0.3,json5:1.0.2,minimatch:3.1.2,wrap-ansi:6.2.0,form-data:4.0.0,type-is:1.6.18,is-empty:1.2.0,mdast-util-mdxjs-esm:2.0.1,toidentifier:1.0.1,pgpass:1.0.5,es-shim-unscopables:1.0.2,@commitlint/types:19.0.3,micromark-extension-mdxjs-esm:3.0.0,is-interactive:2.0.0,@eslint/eslintrc:3.1.0,path-exists:5.0.0,cli-width:4.1.0,typedarray:0.0.6,jackspeak:3.4.3,is-valid-path:0.1.1,esniff:2.0.1,@octokit/request:8.4.0,lowercase-keys:3.0.0,eslint-config-xo:0.44.0,human-signals:2.1.0,pupa:3.1.0,is-proto-prop:2.0.0,cz-conventional-changelog:3.3.0

# also goes on forever....

Hint: Try deleting 'node_modules' and/or 'package-lock.json'.
17:28:35 [Debug] Creating temp build file at: /var/folders/yt/06w44b0n2b52yl7bq__z5sym0000gq/T/jfrog/builds/e7a3c2e18273a65c51013e8460d2a29b84d6b43727c5e52a7e3e673c89122e07
17:28:35 [Debug] Creating temp build file at: /var/folders/yt/06w44b0n2b52yl7bq__z5sym0000gq/T/jfrog/builds/e7a3c2e18273a65c51013e8460d2a29b84d6b43727c5e52a7e3e673c89122e07
17:28:35 [Debug] Restored the file /Users/myusername/Git/--/.npmrc successfully

The temp build file looks like:

Screenshot 2024-09-20 at 5 37 44 PM

When I try a dry-run of the publish build info, I don't see any depedencies being captured.

Reproduction steps

rm -rf node_modules 
npm cache clean --force
jf npm --version # should equal 10.2.4
jq '.lockfileVersion' package-lock.json # should equal 3
jf rt bc "build-prefix/my-app" "v1"
export JFROG_CLI_LOG_LEVEL=DEBUG
jf npm ci --build-name="build-prefix/my-app" --build-number="v1" --threads=1
jf rt bp "build-prefix/my-app" "v1" --env-exclude="*password*;*psw*;*secret*;*key*;*token*;*auth*;" --dry-run

Expected behavior

The JFrog CLI captures the dependencies correctly and the the rt bp dry run displays the correct dependencies.

JFrog CLI version

jf version 2.68.0

Operating system type and version

MacOS Sonoma 14.4.1

JFrog Artifactory version

Cloud SaaS

JFrog Xray version

Cloud SaaS

eric-gonzalez-tfs commented 1 month ago

BTW - I have a copy of my entire terminal transcript that I am willing to share over email.

eric-gonzalez-tfs commented 1 month ago

Hi JFrog team, do you have any recommendations here?

@omerzi @eyalbe4 @yahavi

Or-Geva commented 1 month ago

@eric-gonzalez-tfs, please remove any post or pull scripts from the package.json as they may interfere with the checksum calculations.s.

eric-gonzalez-tfs commented 1 month ago

@eric-gonzalez-tfs, please remove any post or pull scripts from the package.json as they may interfere with the checksum calculations.s.

Hi @Or-Geva - thank you for the details. I can definitely test that and follow up.

Can you confirm if removing preinstall, postinstall, or other lifecycle scripts from package.json is the official JF recommendation? This approach isn't practical as these scripts are crucial for our workflows.

Please advise on a more sustainable solution.

udhay13190 commented 14 hours ago

@eric-gonzalez-tfs Did you get a workaround for this ?

eric-gonzalez-tfs commented 13 hours ago

@eric-gonzalez-tfs Did you get a workaround for this ?

Hi @udhay13190 - I added omit-lockfile-registry-resolved=true to my .npmrc, deleted node_modules, and re-generated the package-lock.json and it has been working consistently since.

Unfortunately, I do not know the repercussions of adding this, and the JF team has not been responsive to my question (above.)