Npm should not run when the push command is used on a bundle that does not contain a NODEJAPP bundlepart

[ChrisAtkinson1]

Npm should not run when the push command is used on a bundle that does not contain a NODEJAPP bundlepart. But it does.

Eg $ cd /u/atkinc/pushtest/NoNode_1.0.3 && rm -r * $ Uploading the bundle to the remote bundle directory

Running npm install for the remote bundle

<$PATH:/usr/lpp/IBM/cnj/IBM/node-latest-os390-s390x/bin" && npm install cics_nodejs_invoke@0.1.0 /u/atkinc/pushtest/NoNode_1.0.3 +-- body-parser@1.18.3 | +-- bytes@3.0.0 | +-- content-type@1.0.4 | +-- debug@2.6.9

Example bundle attached.

NoNodeAPP.zip

[pcoop]

At the moment cics-deploy only runs npm if there's a package.json in the root of the bundle. This bundle has a package.json but it doesn't have a NODEJSAPP... which is a weird combination. Is it important that a bundle which has package.json and doesn't have a NODEJSAPP must not run npm? Might the user in this special case actually prefer than npm was run?

I admit that I was using the presence of package.json as a quick way to infer whether there's a nodejsapp without having to parse the manifest to find out, if this subtlety is important then I can write the extra code to discern between them, but it feels an odd thing to care about. Do we envisage a usage pattern which includes package.json and doesn't include a NODEJSAPP and where not running npm is the required behaviour?

[markcocker]

I would expect zowe cics-bundle push bundle to parse cics.xml to find the list of NODEJSAPPs, then for each change into the directory containing the startscript and run npm install. Maybe npm would allow us to stay in the bundle root, but it could be safer to change directory. From what I can tell on the Mac, npm will give RC=0 if there is no package.json in that directory.

[matthewpwilson]

then for each change into the directory containing the startscript and run npm install

AFAIK npm install needs to run in the directory containing package.json. This might not be the same as the directory containing the startscript

[pcoop]

Fair enough... but this one does feel a bit obscure. What is the use-case we expect for package.json being in the root of the Bundle and the user not wanting an "npm install" to run? Maybe that's a scenario where we should issue an error message and stop as it'll not work as intended.

[pcoop]

Mark's scenario of having more than one package.json within a Bundle is interesting, it's not clear how we should handle that.

There's no 1:1 mapping between package.json files and NODEJSAPPs in the manifest; you could have multiple NODEJSAPPs representing different entry points into the same package with a shared package.json that should only be processed once; or you could have multiple NODEJSAPPs that are isolated from each other each with their own package.json in different directories. If there's more than one package.json then there's no simple way to determine which of them are relevant when there's only a single NODEJSAPP; we either process all of them, or just select whichever subset we think might have been wanted. Of course multiple package.json files within a Bundle is an edge case, it's not something we'd expect anyone to actually want.

So should the algorithm be:

If there are no NODEJSAPPs defined then we don't issue any 'npm install' commands. If there is one or more NODEJSAPPs defined then we issue 'npm install' for every package.json file that is found anywhere within the Bundle.

Or perhaps we'd prefer:

If there are no NODEJSAPPs defined then we don't issue any 'npm install' commands. If there are one or more NODEJSAPPs defined then we tolerate a single package.json in the root of the bundle and issue 'npm install' for it.

Or perhaps:

If there's a package.json then we run 'npm install' for it, regardless of where it is in the Bundle or what's in the manifest.

Or perhaps the current behaviour is most useful:

If there's a package.json in the root of the Bundle, we run 'npm install' for it, regardless of what's in the manifest.

Personally I quite like the simplicity of what we have at the moment: package.json is only supported in PUSH if it's at the root of the Bundle, and if its there then we run npm regardless of what's in the manifest. I can imagine use-cases where power users could use that 'npm install' for scenarios that are nothing to do with node.js... maybe you'd like some server side scripting to run when you deploy your Java bundles... npm could be a way to achieve that.

That said, I also like the idea of simply rejecting a deploy if there's a package.json and no NODEJSAPP... odds are that the user screwed up in some way. We have to tolerate the inverse of course, it's valid for there to be a NODEJSAPP and no package.json. So many options.

[matthewpwilson]

Agreed the behaviour should be: For each package.json in the bundle dir, run npm install once (except don't look for package.jsons inside a node_modules dir) Don't worry about checking if there's a NODEJSAPP

[markcocker]

I successfully pushed a bundle containing two Node.js apps, each within their own directory containing a package.json. npm install was run once in each of those directories.

zowe cics-deploy push bundle --name INVOKE --targetdir /u/cicprov/mnt/CICPY000/bundles --overwrite --verbose
Making remote bundle directory '/u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0'
Accessing contents of the remote bundle directory
Undeploying any existing bundle named 'INVOKE' from CICS
Waiting for JOB79333 to enter OUTPUT (Processing DFHDPLOY UNDEPLOY action)

09:30:28.692539 :  DFHDPLOY CICS TS APPLICATION DEPLOYMENT  2019/04/26 9:30am
09:30:28.693909 :  RELEASE: HCI7300.   SERVICE LEVEL: HCI7300.
09:30:28.695166 :  *
09:30:28.695173 :  SET CICSPLEX(ZOSPTINT);
09:30:30.708069 :  DFHRL2119I Connection to CMAS(CICPY01J) successful.
09:30:32.718057 :  DFHRL2013I Connection to CICSPLEX(ZOSPTINT) successful.
09:30:32.728260 :
09:30:32.728269 :  *
09:30:32.728275 :  UNDEPLOY BUNDLE(INVOKE)
09:30:32.728280 :         SCOPE(CICPY000)
09:30:32.728284 :         STATE(DISCARDED)
09:30:32.728288 :         CSDGROUP(NODE);
09:30:32.728982 :  DFHRL2132I Analyzing CICS regions and CSD attributes.
09:30:39.772016 :  DFHRL2093I BUNDLE(INVOKE) found in SCOPE(CICPY000).
09:30:39.772080 :  DFHRL2129I BUNDLE(INVOKE) state is INSTALLED on 1 CICS regions in SCOPE(CICPY000).
09:30:39.772113 :  DFHRL2129I BUNDLE(INVOKE) state is DISABLED on 1 CICS regions in SCOPE(CICPY000).
09:30:39.772142 :  DFHRL2054I Setting BUNDLE state to DISABLED.
09:30:43.799870 :  DFHRL2042I Discarding BUNDLE(INVOKE).
09:30:52.842568 :  DFHRL2077I BUNDLE(INVOKE) has been discarded from SCOPE(CICPY000).
09:30:54.215325 :  DFHRL2114I Bundle definition for BUNDLE(INVOKE) in CSDGROUP(NODE) has been removed in system(CICPY000).
09:30:54.215388 :  DFHRL2037I UNDEPLOY command successful.
09:30:54.216569 :
09:30:54.216598 :  DFHRL2007I Processing complete.
09:30:54.219108 :  DFHRL2014I Disconnecting from CICSPLEX(ZOSPTINT).

Undeployed existing bundle from CICS
Removing the contents of the remote bundle directory
Issuing SSH command 'rm -r *' in remote directory '/u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0'

$ cd /u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0 && rm -r *
$ 
Uploading the bundle contents to the remote bundle directory
Running 'npm install' in '/u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0/app1'
Issuing SSH command 'export PATH="$PATH:/usr/lpp/IBM/cnj/IBM/node-latest-os390-s390x/bin" && npm install' in remote directory '/u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0/app1'

$ cd /u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0/app1 && export PA
TH="$PATH:/usr/lpp/IBM/cnj/IBM/node-latest-os390-s390x/bin" && npm install
cics_nodejs_invoke@0.1.0 /u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0/app1
+-- body-parser@1.19.0 
| +-- bytes@3.1.0 
| +-- content-type@1.0.4 
| +-- debug@2.6.9 
| | `-- ms@2.0.0 
| +-- depd@1.1.2 
| +-- http-errors@1.7.2 
| | +-- inherits@2.0.3 
| | +-- setprototypeof@1.1.1 
| | +-- statuses@1.5.0 
| | `-- toidentifier@1.0.0 
| +-- iconv-lite@0.4.24 
| | `-- safer-buffer@2.1.2 
| +-- on-finished@2.3.0 
| | `-- ee-first@1.1.1 
| +-- qs@6.7.0 
| +-- raw-body@2.4.0 
| | `-- unpipe@1.0.0 
| `-- type-is@1.6.17 
|   +-- media-typer@0.3.0 
|   `-- mime-types@2.1.24 
|     `-- mime-db@1.40.0 
+-- express@4.16.4 
| +-- accepts@1.3.5 
| | `-- negotiator@0.6.1 
| +-- array-flatten@1.1.1 
| +-- body-parser@1.18.3 
| | +-- bytes@3.0.0 
| | +-- http-errors@1.6.3 
| | +-- iconv-lite@0.4.23 
| | `-- raw-body@2.3.3 
| +-- content-disposition@0.5.2 
| +-- cookie@0.3.1 
| +-- cookie-signature@1.0.6 
| +-- encodeurl@1.0.2 
| +-- escape-html@1.0.3 
| +-- etag@1.8.1 
| +-- finalhandler@1.1.1 
| | `-- statuses@1.4.0 
| +-- fresh@0.5.2 
| +-- merge-descriptors@1.0.1 
| +-- methods@1.1.2 
| +-- parseurl@1.3.3 
| +-- path-to-regexp@0.1.7 
| +-- proxy-addr@2.0.5 
| | +-- forwarded@0.1.2 
| | `-- ipaddr.js@1.9.0 
| +-- qs@6.5.2 
| +-- range-parser@1.2.0 
| +-- safe-buffer@5.1.2 
| +-- send@0.16.2 
| | +-- destroy@1.0.4 
| | +-- http-errors@1.6.3 
| | | `-- setprototypeof@1.1.0 
| | +-- mime@1.4.1 
| | `-- statuses@1.4.0 
| +-- serve-static@1.13.2 
| +-- setprototypeof@1.1.0 
| +-- statuses@1.4.0 
| +-- utils-merge@1.0.1 
| `-- vary@1.1.2 
`-- ibm-cics-api@1.0.1 
  +-- request@2.88.0 
  | +-- aws-sign2@0.7.0 
  | +-- aws4@1.8.0 
  | +-- caseless@0.12.0 
  | +-- combined-stream@1.0.7 
  | | `-- delayed-stream@1.0.0 
  | +-- extend@3.0.2 
  | +-- forever-agent@0.6.1 
  | +-- form-data@2.3.3 
  | | `-- asynckit@0.4.0 
  | +-- har-validator@5.1.3 
  | | +-- ajv@6.10.0 
  | | | +-- fast-deep-equal@2.0.1 
  | | | +-- fast-json-stable-stringify@2.0.0 
  | | | +-- json-schema-traverse@0.4.1 
  | | | `-- uri-js@4.2.2 
  | | |   `-- punycode@2.1.1 
  | | `-- har-schema@2.0.0 
  | +-- http-signature@1.2.0 
  | | +-- assert-plus@1.0.0 
  | | +-- jsprim@1.4.1 
  | | | +-- extsprintf@1.3.0 
  | | | +-- json-schema@0.2.3 
  | | | `-- verror@1.10.0 
  | | |   `-- core-util-is@1.0.2 
  | | `-- sshpk@1.16.1 
  | |   +-- asn1@0.2.4 
  | |   +-- bcrypt-pbkdf@1.0.2 
  | |   +-- dashdash@1.14.1 
  | |   +-- ecc-jsbn@0.1.2 
  | |   +-- getpass@0.1.7 
  | |   +-- jsbn@0.1.1 
  | |   `-- tweetnacl@0.14.5 
  | +-- is-typedarray@1.0.0 
  | +-- isstream@0.1.2 
  | +-- json-stringify-safe@5.0.1 
  | +-- oauth-sign@0.9.0 
  | +-- performance-now@2.1.0 
  | +-- qs@6.5.2 
  | +-- tough-cookie@2.4.3 
  | | +-- psl@1.1.31 
  | | `-- punycode@1.4.1 
  | +-- tunnel-agent@0.6.0 
  | `-- uuid@3.3.2 
  `-- semver@5.7.0 

$ 
Running 'npm install' in '/u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0/app2'
Issuing SSH command 'export PATH="$PATH:/usr/lpp/IBM/cnj/IBM/node-latest-os390-s390x/bin" && npm install' in remote directory '/u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0/app2'

$ cd /u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0/app2 && export PA
TH="$PATH:/usr/lpp/IBM/cnj/IBM/node-latest-os390-s390x/bin" && npm install
cics_nodejs_invoke@0.1.0 /u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0/app2
+-- body-parser@1.19.0 
| +-- bytes@3.1.0 
| +-- content-type@1.0.4 
| +-- debug@2.6.9 
| | `-- ms@2.0.0 
| +-- depd@1.1.2 
| +-- http-errors@1.7.2 
| | +-- inherits@2.0.3 
| | +-- setprototypeof@1.1.1 
| | +-- statuses@1.5.0 
| | `-- toidentifier@1.0.0 
| +-- iconv-lite@0.4.24 
| | `-- safer-buffer@2.1.2 
| +-- on-finished@2.3.0 
| | `-- ee-first@1.1.1 
| +-- qs@6.7.0 
| +-- raw-body@2.4.0 
| | `-- unpipe@1.0.0 
| `-- type-is@1.6.17 
|   +-- media-typer@0.3.0 
|   `-- mime-types@2.1.24 
|     `-- mime-db@1.40.0 
+-- express@4.16.4 
| +-- accepts@1.3.5 
| | `-- negotiator@0.6.1 
| +-- array-flatten@1.1.1 
| +-- body-parser@1.18.3 
| | +-- bytes@3.0.0 
| | +-- http-errors@1.6.3 
| | +-- iconv-lite@0.4.23 
| | `-- raw-body@2.3.3 
| +-- content-disposition@0.5.2 
| +-- cookie@0.3.1 
| +-- cookie-signature@1.0.6 
| +-- encodeurl@1.0.2 
| +-- escape-html@1.0.3 
| +-- etag@1.8.1 
| +-- finalhandler@1.1.1 
| | `-- statuses@1.4.0 
| +-- fresh@0.5.2 
| +-- merge-descriptors@1.0.1 
| +-- methods@1.1.2 
| +-- parseurl@1.3.3 
| +-- path-to-regexp@0.1.7 
| +-- proxy-addr@2.0.5 
| | +-- forwarded@0.1.2 
| | `-- ipaddr.js@1.9.0 
| +-- qs@6.5.2 
| +-- range-parser@1.2.0 
| +-- safe-buffer@5.1.2 
| +-- send@0.16.2 
| | +-- destroy@1.0.4 
| | +-- http-errors@1.6.3 
| | | `-- setprototypeof@1.1.0 
| | +-- mime@1.4.1 
| | `-- statuses@1.4.0 
| +-- serve-static@1.13.2 
| +-- setprototypeof@1.1.0 
| +-- statuses@1.4.0 
| +-- utils-merge@1.0.1 
| `-- vary@1.1.2 
`-- ibm-cics-api@1.0.1 
  +-- request@2.88.0 
  | +-- aws-sign2@0.7.0 
  | +-- aws4@1.8.0 
  | +-- caseless@0.12.0 
  | +-- combined-stream@1.0.7 
  | | `-- delayed-stream@1.0.0 
  | +-- extend@3.0.2 
  | +-- forever-agent@0.6.1 
  | +-- form-data@2.3.3 
  | | `-- asynckit@0.4.0 
  | +-- har-validator@5.1.3 
  | | +-- ajv@6.10.0 
  | | | +-- fast-deep-equal@2.0.1 
  | | | +-- fast-json-stable-stringify@2.0.0 
  | | | +-- json-schema-traverse@0.4.1 
  | | | `-- uri-js@4.2.2 
  | | |   `-- punycode@2.1.1 
  | | `-- har-schema@2.0.0 
  | +-- http-signature@1.2.0 
  | | +-- assert-plus@1.0.0 
  | | +-- jsprim@1.4.1 
  | | | +-- extsprintf@1.3.0 
  | | | +-- json-schema@0.2.3 
  | | | `-- verror@1.10.0 
  | | |   `-- core-util-is@1.0.2 
  | | `-- sshpk@1.16.1 
  | |   +-- asn1@0.2.4 
  | |   +-- bcrypt-pbkdf@1.0.2 
  | |   +-- dashdash@1.14.1 
  | |   +-- ecc-jsbn@0.1.2 
  | |   +-- getpass@0.1.7 
  | |   +-- jsbn@0.1.1 
  | |   `-- tweetnacl@0.14.5 
  | +-- is-typedarray@1.0.0 
  | +-- isstream@0.1.2 
  | +-- json-stringify-safe@5.0.1 
  | +-- oauth-sign@0.9.0 
  | +-- performance-now@2.1.0 
  | +-- qs@6.5.2 
  | +-- tough-cookie@2.4.3 
  | | +-- psl@1.1.31 
  | | `-- punycode@1.4.1 
  | +-- tunnel-agent@0.6.0 
  | `-- uuid@3.3.2 
  `-- semver@5.7.0 

$ 
Deploying bundle 'INVOKE' to CICS
Waiting for JOB79354 to enter OUTPUT (Processing DFHDPLOY DEPLOY action)

09:32:16.160646 :  DFHDPLOY CICS TS APPLICATION DEPLOYMENT  2019/04/26 9:32am
09:32:16.161316 :  RELEASE: HCI7300.   SERVICE LEVEL: HCI7300.
09:32:16.162539 :  *
09:32:16.162546 :  SET CICSPLEX(ZOSPTINT);
09:32:18.182547 :  DFHRL2119I Connection to CMAS(CICPY01J) successful.
09:32:20.198679 :  DFHRL2013I Connection to CICSPLEX(ZOSPTINT) successful.
09:32:20.209237 :
09:32:20.209247 :  *
09:32:20.209253 :  DEPLOY BUNDLE(INVOKE)
09:32:20.209257 :         BUNDLEDIR(/u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0
09:32:20.209262 :          .0)
09:32:20.209266 :         SCOPE(CICPY000)
09:32:20.209270 :         STATE(ENABLED)
09:32:20.209273 :         CSDGROUP(NODE);
09:32:20.212894 :  DFHRL2132I Analyzing CICS regions and CSD attributes.
09:32:28.270985 :  DFHRL2051I Creating BUNDLE definition on the CSD in system(CICPY000).
09:32:29.637610 :  DFHRL2049I Creating BUNDLE definition.
09:32:29.642318 :  DFHRL2024I BUNDLE definition(INVOKE) VERSION(1) successfully created.
09:32:30.646350 :  DFHRL2052I Installing BUNDLE definition.
09:32:30.647279 :  DFHRL2131I Waiting for BUNDLE(INVOKE) to be installed in active regions in SCOPE(CICPY000).
09:32:46.728320 :  DFHRL2130I BUNDLE(INVOKE) installed in 1 of 1 regions in SCOPE(CICPY000).
09:32:46.728365 :  DFHRL2054I Setting BUNDLE state to ENABLED.
09:32:50.754049 :  DFHRL2012I DEPLOY command completed successfully.
09:32:50.756132 :
09:32:50.756161 :  DFHRL2007I Processing complete.
09:32:50.759573 :  DFHRL2014I Disconnecting from CICSPLEX(ZOSPTINT).

Deployed bundle 'INVOKE' to CICS
PUSH operation completed.

Assigning back to @ChrisAtkinson1 incase he wants to run further tests.

[ChrisAtkinson1]

Tried on my bundle with 4 apps and 5 .json files. 1 in top directory - apps in sub directories - worked as above.