search

jfrog / jfrog-azure-devops-extension

Apache License 2.0
47 stars 62 forks source link

Not able to publish multiple npm packages in a monorepo #473

Closed gavintoddchurch closed 6 months ago

gavintoddchurch commented 8 months ago

Describe the bug

In our azure devops pipeline we are using JFrogNPM@1 task to publish to our artifactory registry

When we do this only the last package is actually uploaded to the registry. It performs the pack on each of them but only publishes the last.

Current behavior


npm notice total files:   5                                                         
npm notice 
npm notice 
npm notice 📦  @<redacted>/oauth-client-secrets-manager@1.0.13-alpha.0
npm notice === Tarball Contents === 
npm notice 302B  constructor-schema.js          
npm notice 2.1kB index.js                       
npm notice 545B  package.json                   
npm notice 573B  secret-manager.js              
npm notice 1.4kB test/constructor-schema.test.js
npm notice 4.0kB test/index.test.js             
npm notice 1.3kB test/secret-manager.test.js    
npm notice === Tarball Details === 
npm notice name:          @<redacted>/oauth-client-secrets-manager                  
npm notice version:       1.0.13-alpha.0                                                     
npm notice filename:      <redacted>-oauth-client-secrets-manager-1.0.13-alpha.0.tgz
npm notice package size:  2.8 kB                                                             
npm notice unpacked size: 10.2 kB                                                            
npm notice shasum:        1e548569988290eb97352b2541b2676aaa0caa13                           
npm notice integrity:     sha512-flbTPTTThQJJU[...]4P/lhQqBfKQmg==                           
npm notice total files:   7                                                                  
npm notice 
npm notice 
npm notice 📦  @<redacted>/oauth-client-static@1.0.13-alpha.0
npm notice === Tarball Contents === 
npm notice 222B  constructor-schema.js          
npm notice 406B  index.js                       
npm notice 474B  package.json                   
npm notice 1.1kB test/constructor-schema.test.js
npm notice 1.6kB test/index.test.js             
npm notice === Tarball Details === 
npm notice name:          @<redacted>/oauth-client-static                  
npm notice version:       1.0.13-alpha.0                                            
npm notice filename:      <redacted>-oauth-client-static-1.0.13-alpha.0.tgz
npm notice package size:  1.5 kB                                                    
npm notice unpacked size: 3.8 kB                                                    
npm notice shasum:        c108b360f66053dedf7482f40711c619a38ab668                  
npm notice integrity:     sha512-j3ZLGDuneZ29j[...]GV6odEKAD24rg==                  
npm notice total files:   5                                                         
npm notice 
20:34:02 [Info] [Thread 2] Uploading: /agent/_work/38/s/javascript/<redacted>-oauth-client-static-1.0.13-alpha.0.tgz
20:34:03 [Info] npm publish finished successfully.
{
  "status": "success",
  "totals": {
    "success": 0,
    "failure": 0
  }
}

Reproduction steps

No response

Expected behavior

We expect it to publish all 3 packages to the registry.

Azure DevOps extension name and version

JFrog npm 1.9.2

JFrog CLI version

JFrog CLI version: 2.52.9

Operating system type and version

Ubuntu 22.04.3 LTS

JFrog Artifactory version (if relevant)

No response

JFrog Xray version (if relevant)

No response

JFrog Distribution version (if relevant)

No response

gavintoddchurch commented 7 months ago

Any ideas on this?

gavintoddchurch commented 7 months ago

@sverdlov93 Any ideas on this?

sverdlov93 commented 7 months ago

Hi @GavinToddlds, Can you please try again with JFROG_CLI_LOG_LEVEL=DEBUG environment variable and share the full task logs? You should be able to see there the exact npm command used by this task and try it locally outside of JFrogNPM task and check if it works.

gavintoddchurch commented 7 months ago

Here is the command jf npm p --workspaces If we try just npm p --workspaces it works but through jf cli it only publishes the last one.

Here is the logs

Starting: Publish to npm registry
==============================================================================
Task         : JFrog npm
Description  : Install, pack and publish npm packages from and to Artifactory while allowing to collect build-info. The collected build-info can be later published to Artifactory by the "JFrog Publish Build Info" task.
Version      : 1.9.2
Author       : JFrog
Help         : [More Information](https://github.com/jfrog/jfrog-azure-devops-extension#JFrog-npm-Task)
==============================================================================
##[debug]Using node path: /agent/externals/node10/bin/node
##[debug]agent.TempDirectory=/agent/_work/_temp
##[debug]loading inputs and endpoints
##[debug]loading INPUT_COMMAND
##[debug]loading INPUT_ARTIFACTORYCONNECTION
##[debug]loading INPUT_TARGETREPO
##[debug]loading INPUT_WORKINGFOLDER
##[debug]loading INPUT_ARGUMENTS
##[debug]loading INPUT_COLLECTBUILDINFO
##[debug]loading INPUT_THREADS
##[debug]loading INPUT_BUILDNAME
##[debug]loading INPUT_BUILDNUMBER
##[debug]loading INPUT_INCLUDEENVVARS
##[debug]loading ENDPOINT_AUTH_<redacted>
##[debug]loading ENDPOINT_AUTH_SCHEME_<redacted>
##[debug]loading ENDPOINT_AUTH_PARAMETER_<redacted>_USERNAME
##[debug]loading ENDPOINT_AUTH_PARAMETER_<redacted>_PASSWORD
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
##[debug]loaded 18
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]check path : /agent/_work/_tasks/JFrogNpm_3a5a35c4-b755-4821-b4d9-f10ce6e1f86c/1.9.2/node_modules/azure-pipelines-tool-lib/lib.json
##[debug]adding resource file: /agent/_work/_tasks/JFrogNpm_3a5a35c4-b755-4821-b4d9-f10ce6e1f86c/1.9.2/node_modules/azure-pipelines-tool-lib/lib.json
##[debug]system.culture=en-US
##[debug]check path : /agent/_work/_tasks/JFrogNpm_3a5a35c4-b755-4821-b4d9-f10ce6e1f86c/1.9.2/node_modules/azure-pipelines-tasks-java-common/lib.json
##[debug]adding resource file: /agent/_work/_tasks/JFrogNpm_3a5a35c4-b755-4821-b4d9-f10ce6e1f86c/1.9.2/node_modules/azure-pipelines-tasks-java-common/lib.json
##[debug]system.culture=en-US
##[debug]Agent.ToolsDirectory=/opt/hostedtoolcache
##[debug]JFROG_CLI_PIPELINE_REQUESTED_VERSION_AZURE=undefined
##[debug]isExplicit: 2.52.9
##[debug]explicit? true
##[debug]Agent.Version=3.232.3
##[debug]Agent.ToolsDirectory=/opt/hostedtoolcache
##[debug]checking cache: /opt/hostedtoolcache/jf/2.52.9/x64
Found tool in cache: jf 2.52.9 x64
##[debug]Using existing versioned cli path: /opt/hostedtoolcache/jf/2.52.9/x64/jf
Running jfrog-cli from /opt/hostedtoolcache/jf/2.52.9/x64/jf
JFrog CLI version: 2.52.9
##[debug]set JFROG_CLI_TASK_SELECTED_VERSION_AZURE=2.52.9
##[debug]Processed: ##vso[task.setvariable variable=JFROG_CLI_TASK_SELECTED_VERSION_AZURE;isOutput=false;issecret=false;]2.52.9
##[debug]System.DefaultWorkingDirectory=/agent/_work/38/s
##[debug]workingFolder=/agent/_work/38/s/javascript
##[debug]command=pack and publish
##[debug]command=pack and publish
##[debug]artifactoryConnection=<redacted>
##[debug]Build.DefinitionName=<redacted>.pws-oauth-client
##[debug]Build.BuildNumber=20240123.54
##[debug]<redacted>=https://<redacted>.<redacted>.org/artifactory
##[debug]<redacted> auth param username = ***
##[debug]<redacted> auth param password = ***
##[debug]<redacted> auth param apitoken = undefined
##[debug]JFROG_CLI_TASK_SELECTED_VERSION_AZURE=2.52.9
Executing JFrog CLI Command:
/opt/hostedtoolcache/jf/2.52.9/x64/jf c add "<redacted>.pws-oauth-client_20240123.54_npmpack and publish_deployer_1706042040855" --artifactory-url="https://<redacted>.<redacted>.org/artifactory" --interactive=false --user='***' --basic-auth-only --password-stdin
Executing JFrog CLI Command:
/opt/hostedtoolcache/jf/2.52.9/x64/jf c use "<redacted>.pws-oauth-client_20240123.54_npmpack and publish_deployer_1706042040855"
20:34:01 [Info] Using server ID '<redacted>.pws-oauth-client_20240123.54_npmpack and publish_deployer_1706042040855' (https://<redacted>.<redacted>.org/artifactory/)
##[debug]targetRepo=npm-ics
Executing JFrog CLI Command:
/opt/hostedtoolcache/jf/2.52.9/x64/jf npmc --server-id-deploy="<redacted>.pws-oauth-client_20240123.54_npmpack and publish_deployer_1706042040855" --repo-deploy="npm-ics"
20:34:01 [Info] npm build config successfully created.
##[debug]arguments=--workspaces
##[debug]collectBuildInfo=false
Executing JFrog CLI Command:
/opt/hostedtoolcache/jf/2.52.9/x64/jf npm p --workspaces
20:34:01 [Info] Running npm Publish
npm notice 
npm notice 📦  @<redacted>/oauth-client-common@1.0.13-alpha.0
npm notice === Tarball Contents === 
npm notice 520B  constructor-schema.js          
npm notice 5.0kB index.js                       
npm notice 508B  package.json                   
npm notice 756B  test/constructor-schema.test.js
npm notice 3.3kB test/index.test.js             
npm notice === Tarball Details === 
npm notice name:          @<redacted>/oauth-client-common                  
npm notice version:       1.0.13-alpha.0                                            
npm notice filename:      <redacted>-oauth-client-common-1.0.13-alpha.0.tgz
npm notice package size:  3.1 kB                                                    
npm notice unpacked size: 10.1 kB                                                   
npm notice shasum:        5d7674821ea56ca57716204c3ef79e04495c92fe                  
npm notice integrity:     sha512-vT00QVMgvdRrk[...]KlhdEg5YHgLeg==                  
npm notice total files:   5                                                         
npm notice 
npm notice 
npm notice 📦  @<redacted>/oauth-client-secrets-manager@1.0.13-alpha.0
npm notice === Tarball Contents === 
npm notice 302B  constructor-schema.js          
npm notice 2.1kB index.js                       
npm notice 545B  package.json                   
npm notice 573B  secret-manager.js              
npm notice 1.4kB test/constructor-schema.test.js
npm notice 4.0kB test/index.test.js             
npm notice 1.3kB test/secret-manager.test.js    
npm notice === Tarball Details === 
npm notice name:          @<redacted>/oauth-client-secrets-manager                  
npm notice version:       1.0.13-alpha.0                                                     
npm notice filename:      <redacted>-oauth-client-secrets-manager-1.0.13-alpha.0.tgz
npm notice package size:  2.8 kB                                                             
npm notice unpacked size: 10.2 kB                                                            
npm notice shasum:        1e548569988290eb97352b2541b2676aaa0caa13                           
npm notice integrity:     sha512-flbTPTTThQJJU[...]4P/lhQqBfKQmg==                           
npm notice total files:   7            
npm notice 
npm notice 
npm notice 📦  @<redacted>/oauth-client-static@1.0.13-alpha.0
npm notice === Tarball Contents === 
npm notice 222B  constructor-schema.js          
npm notice 406B  index.js                       
npm notice 474B  package.json                   
npm notice 1.1kB test/constructor-schema.test.js
npm notice 1.6kB test/index.test.js             
npm notice === Tarball Details === 
npm notice name:          @<redacted>/oauth-client-static                  
npm notice version:       1.0.13-alpha.0                                            
npm notice filename:      <redacted>-oauth-client-static-1.0.13-alpha.0.tgz
npm notice package size:  1.5 kB                                                    
npm notice unpacked size: 3.8 kB                                                    
npm notice shasum:        c108b360f66053dedf7482f40711c619a38ab668                  
npm notice integrity:     sha512-j3ZLGDuneZ29j[...]GV6odEKAD24rg==                  
npm notice total files:   5                                                         
npm notice 
20:34:02 [Info] [Thread 2] Uploading: /agent/_work/38/s/javascript/<redacted>-oauth-client-static-1.0.13-alpha.0.tgz
20:34:03 [Info] npm publish finished successfully.
{
  "status": "success",
  "totals": {
    "success": 0,
    "failure": 0
  }
}
##[debug]task result: Succeeded
##[debug]Processed: ##vso[task.complete result=Succeeded;]Build Succeeded.
Executing JFrog CLI Command:
/opt/hostedtoolcache/jf/2.52.9/x64/jf c remove "<redacted>.pws-oauth-client_20240123.54_npmpack and publish_deployer_1706042040855" --quiet
##[debug]Removing JFrog CLI build tool configuration...
##[debug]rm -rf /agent/_work/38/s/javascript/.jfrog/projects
##[debug]removing directory
##[debug]includeEnvVars=false
EyalDelarea commented 7 months ago

Hey @GavinToddlds,

Thanks for bringing this issue to our attention! Your observation was spot on – our publish command was failing to handle multiple workspaces. I've submitted a PR to address this in thejfrog-cli-core repository. And I will update you here once the change will take effect in the extension.

Appreciate your contribution.

EyalDelarea commented 6 months ago

Hey @GavinToddlds, A new version has been released v2.9.4 using a new version of the JFrog CLI v2.54.0 And this issue should be resolved. Waiting for your feedback!

gavintoddchurch commented 6 months ago

@EyalDelarea That did fix the issue. Thank you! I'll close this issue.