search

SAP-archive / cloud-s4-sdk-pipeline

The Cloud SDK pipeline uses the Cloud SDK continuous delivery server for building, checking, and deploying extension applications. Projects based on the SAP Cloud SDK archetype will automatically use this pipeline.
Apache License 2.0
65 stars 25 forks source link

Does cache of package.json exist in Docker? How to clear it? #22

Closed Wesley-Qi closed 4 years ago

Wesley-Qi commented 4 years ago

Hi colleagues,

We met a strange issue in pipeline setting. In root folder of our project, there is a package.json file. Previously it contains content like:

"cds": {
        "requires": {
            "db": {
                "kind": "hana",
                "model": [
                    "db",
                    "db_master"
                ]
            }
        }
    }

However we removed value "db_master" and committed a new version:

"cds": {
        "requires": {
            "db": {
                "kind": "hana",
                "model": [
                    "db"
                ]
            }
        }
    }

This commit triggers the pipeline, however npmAudit step failed because of the following error:

[2019-10-16T09:53:39.666Z] --- BEGIN LIBRARY STEP: checkNpmAudit.groovy ---
[2019-10-16T09:53:41.273Z] Expected npm package lock file to exist. This is a requirement for npm audit. See https://docs.npmjs.com/files/package-locks for background. Executing `npm install` to create a package-lock.json at 'db'
[2019-10-16T09:53:41.587Z] --- BEGIN LIBRARY STEP: executeNpm.groovy ---
[2019-10-16T09:53:44.223Z] [handlePipelineStepErrors] Configuration: [collectTelemetryData:false, changeManagement:[type:NONE, transportRequestLabel:TransportRequest\s?:, changeDocumentLabel:ChangeDocument\s?:, clientOpts:, credentialsId:CM, git:[from:origin/master, to:HEAD, format:%b], solman:[docker:[image:ppiper/cm-client, options:[], envVars:[:], pullImage:true]], cts:[docker:[image:ppiper/cm-client, options:[], envVars:[:], pullImage:true]], rfc:[docker:[image:rfc, options:[], envVars:[:], pullImage:true]]], githubApiUrl:https://api.github.com, githubServerUrl:https://github.com, gitSshKeyCredentialsId:SSH_private_key_of_technical_user_TUGHCALMBuild, jenkinsKubernetes:[jnlpAgent:docker.wdf.sap.corp:51162/s4sdk/jenkins-agent-k8s-sap], manualConfirmation:true, manualConfirmationMessage:Shall we proceed to Promote & Release?, manualConfirmationTimeout:720, productiveBranch:NAASTest, whitesource:[serviceUrl:https://saas.whitesourcesoftware.com/api], unsafeMode:true, defaultNode:, automaticVersioning:false, echoDetails:true, failOnError:true, libraryDocumentationUrl:https://sap.github.io/jenkins-library/, libraryRepositoryUrl:https://github.com/SAP/jenkins-library/, mandatorySteps:[], stepTimeouts:[:], extensionRepository:https://github.wdf.sap.corp/CALMBuild/pipeline-extensions, xMake:[buildQuality:Milestone, novaCredentialsId:xmakeNova], gitCredentialsId:SSH_private_key_of_technical_user_TUGHCALMBuild, buildTool:mta, deployTool:mtaDeployPlugin, mtaDeployParameters:-f --version-rule ALL, cfApiEndpoint:https://api.cf.sap.hana.ondemand.com, cfOrg:cxs-calmdevprovider, cfCredentialsId:Cloud_Cockpit_login_with_technical_user_TUGHCALMBuild, influxDBServer:Jenkins, verbose:true, gitUrl:https://github.wdf.sap.corp/CALMBuild/ptm-pjm-srv.git, projectName:com.sap.calm.imp.pjm, gitCommitId:ced6a84f00b7e3fee7f7adab82ccb487d0b96d38, stepName:dockerExecute, stepParameters:[script:Script1@5abdf1ab, dockerImage:ppiper/node-browsers, dockerOptions:[--cap-add=SYS_ADMIN]], stepNameDoc:dockerExecute]
[2019-10-16T09:53:44.488Z] --- Begin library step of: dockerExecute ---
[2019-10-16T09:53:44.987Z] [dockerExecute] Sending telemetry data is disabled.
[2019-10-16T09:53:46.176Z] [INFO][dockerExecute] Executing inside a Kubernetes Container.
[2019-10-16T09:53:48.044Z] + npm install
[2019-10-16T09:54:00.410Z] 
[2019-10-16T09:54:00.425Z] > @sap/hana-client@2.4.144 install /home/jenkins/workspace/CALMBuild_ptm-pjm-srv_PR-174/db/node_modules/@sap/hdi-deploy/node_modules/@sap/hana-client
[2019-10-16T09:54:00.425Z] > node build.js
[2019-10-16T09:54:00.425Z] 
[2019-10-16T09:54:00.425Z] 
[2019-10-16T09:54:00.425Z] > @sap/hana-client@2.4.139 install /home/jenkins/workspace/CALMBuild_ptm-pjm-srv_PR-174/db/node_modules/@sap/hdi-dynamic-deploy/node_modules/@sap/hdi-deploy/node_modules/@sap/hana-client
[2019-10-16T09:54:00.425Z] > node build.js
[2019-10-16T09:54:00.425Z] 
[2019-10-16T09:54:00.694Z] 
[2019-10-16T09:54:00.714Z] > deploy@ postinstall /home/jenkins/workspace/CALMBuild_ptm-pjm-srv_PR-174/db
[2019-10-16T09:54:00.714Z] > node .build.js
[2019-10-16T09:54:00.714Z] 
[2019-10-16T09:54:04.058Z] npm notice created a lockfile as package-lock.json. You should commit this file.
[2019-10-16T09:54:04.101Z] npm WARN pjm-srv@1.0.0 license should be a valid SPDX license expression
[2019-10-16T09:54:04.101Z] 
[2019-10-16T09:54:04.101Z] added 81 packages from 111 contributors and audited 174 packages in 2.733s
[2019-10-16T09:54:04.101Z] found 0 vulnerabilities
[2019-10-16T09:54:04.101Z] 
[2019-10-16T09:54:04.686Z] 
[2019-10-16T09:54:04.703Z] > pjm-srv@1.0.0 build /home/jenkins/workspace/CALMBuild_ptm-pjm-srv_PR-174
[2019-10-16T09:54:04.703Z] > cds build/all --clean
[2019-10-16T09:54:04.703Z] 
[2019-10-16T09:54:04.703Z] 
[2019-10-16T09:54:04.703Z] [cds] - cds [3.16.3], compiler [1.17.2], home [/home/jenkins/workspace/CALMBuild_ptm-pjm-srv_PR-174/node_modules/@sap/cds]
[2019-10-16T09:54:04.703Z] 
[2019-10-16T09:54:04.703Z] {
[2019-10-16T09:54:04.703Z]   "build": {
[2019-10-16T09:54:04.703Z]     "target": ".",
[2019-10-16T09:54:04.703Z]     "tasks": [
[2019-10-16T09:54:04.703Z]       {"for":"hana","src":"db","options":{"model":["db","srv"]}},
[2019-10-16T09:54:04.703Z]       {"for":"hana","src":"db_master","options":{"model":["db_master"]}},

You may see the last line, that it can still read the REMOVED value "db_master", and it will cause error in the following:

[2019-10-16T09:54:04.703Z] Error: Build task [@sap/cds/bin/build/hana] could not be resolved - folder src [/home/jenkins/workspace/CALMBuild_ptm-pjm-srv_PR-174/db_master] does not exist

I'm wondering whether the build still uses the old version of package.json file? If yes, is there a way to ask pipeline to use the new file everytime rather than using the one which is cached?

Debug log as the following:

7. Unsuccessful Build

The build failed in the npmAudit stage.

The following error message was generated:

hudson.AbortException: script returned exit code 1

The following stack trace was generated:

org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.handleExit(DurableTaskStep.java:569)
org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.check(DurableTaskStep.java:515)
org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.run(DurableTaskStep.java:461)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

Thanks a lot, Terry

fwilhe commented 4 years ago

For those who face the same issue: Look for a file called .cdsrc.json which still might contain the old state.