Deploying Hosted Target Proxy without node_modules AND preserving-policies

[classicalguss]

I have an existing Hosted Target proxy. I'm trying to find a way to deploy changes without including node_modules and also keep the other Policies.

If I try to use deployproxy with --resolve-modules like the following code apigeetool deployproxy --resolve-modules --debug --verbose -u ***** -o maf-holding -e dev -n DKInternal-Share-Hosted -p *****

I get the following error

Running "npm install" at Apigee. This may take several minutes.
NPM failed with 500
npm ERR! install Couldn't read dependencies
npm ERR! Trireme Trireme
npm ERR! argv "./node" "/opt/apigee/edge-management-server-20.01.06-0.0.20373/lib/node_modules/npm/cli.js" "install" "--registry" "https://registry.npmjs.org/" "--cache" "/opt/apigee/var/log/edge-management-server/work/npm-cache" "--ignore-scripts" "true" "--bin-links" "false" "--production"
npm ERR! node v0.10.32
npm ERR! npm  v2.14.1
npm ERR! path /opt/apigee/var/log/edge-management-server/work/npm/npm5821663121571056737.tmp/package.json
npm ERR! code ENOPACKAGEJSON
npm ERR! errno -2

npm ERR! package.json ENOENT:/opt/apigee/var/log/edge-management-server/work/npm/npm5821663121571056737.tmp/package.json
npm ERR! package.json This is most likely not a problem with npm itself.
npm ERR! package.json npm can't find a package.json file in your current directory.

npm ERR! Please include the following file with any support request:
npm ERR!     /opt/apigee/var/log/edge-management-server/work/npm/npm5821663121571056737.tmp/npm-debug.log

Error: NPM install failed with status code 500
    at Request._callback (/usr/local/lib/node_modules/apigeetool/lib/commands/deployproxy.js:673:14)
    at Request.self.callback (/usr/local/lib/node_modules/apigeetool/node_modules/request/request.js:185:22)
    at Request.emit (events.js:209:13)
    at Request.<anonymous> (/usr/local/lib/node_modules/apigeetool/node_modules/request/request.js:1161:10)
    at Request.emit (events.js:209:13)
    at IncomingMessage.<anonymous> (/usr/local/lib/node_modules/apigeetool/node_modules/request/request.js:1083:12)
    at Object.onceWrapper (events.js:298:28)
    at IncomingMessage.emit (events.js:214:15)
    at endReadableNT (_stream_readable.js:1178:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21)

Note: if I go to edge and check. I find that the proxy got uploaded but not deployed

Now if I try to go to the hosted target folder /resources/hosted and run deployhostedtarget with --preserve-policies such as the following:

apigeetool deployhostedtarget --preserve-policies -u ***** -o maf-holding -e dev -n DKInternal-Share-Hosted -d . -p ***** Note: I don't seem to notice any --resolve-modules flag for this particular command.

I see that the node_modules have been included in the proxy when I go to Apigee Edge

I also tried using deploynodeapp but I get "file conflicting errors"

Hope anyone can help.

[whitlockjc]

Looking at the error, this is what I see that stands out: "package.json npm can't find a package.json file in your current directory." For the -d ., is there a package.json in there?

[classicalguss]

Thanks for the reply @whitlockjc

Looking at the error, this is what I see that stands out: "package.json npm can't find a package.json file in your current directory." For the -d ., is there a package.json in there?

No, the package.json exists in the following path: ./apiproxy/resources/hosted/package.json

If I changed -d to where the package.json is, I wouldn't be able to use deployproxy. deployproxy only works when you're in a directory that has apiproxy/

[whitlockjc]

You shouldn't be using deployproxy with the --resolve-modules because that is a Trireme-only feature. Unfortunately, using deployproxy will not handle your hosted/node_modules correctly so you should be using deployhostedtarget. Using deployhostedtarget should not muck with your existing policies and it is the only way to deploy a Hosted Target with all of the special treatment of node_modules.

I'm going to cc @AdamMagaluk as he wrote this code.

[classicalguss]

@whitlockjc But the problem with deployhostedtarget is that it's including node_modules in the upload.

And also if I had the complete proxy with all the policies in a workspace, I wouldn't be able to upload any changes on the policies (I'd be forced to change stuff on Apigee Edge)

[whitlockjc]

Can you give me your deployhostedtarget command? Also, what version of apigeetool are you using? In my opinion, there are two issues here:

1. deployhostedtarget should not be uploading node_modules
2. deployproxy should work with Hosted Targets

[classicalguss]

I had it in the original comment, but erroneously included -U false Anyway here it is again. apigeetool deployhostedtarget --preserve-policies -u ***** -o maf-holding -e dev -n DKInternal-Share-Hosted -d . -p *****

My apigeetool version is 0.14.2

[whitlockjc]

That should not include node_modules. But I think I have an idea of why it appears that way. My assumption, and I could be wrong as I'm still tracking this down, is that at one point you used deployproxy and it uploaded node_modules. Any chance that is the case? A simple test could be performed where you delete node_modules manually and then issue your deployhostdtarget command again.

[classicalguss]

Yes. I thought of that a few minutes ago. I'm currently testing it. Will let you know.

However, there would still be the issue of trying to deploy a hosted target proxy when you change something in the attached Policies.

[classicalguss]

@whitlockjc I've ran my tests and here is what I exactly did:

1. Ran deployhostedapp on a non-existing proxy apigeetool deployhostedtarget --preserve-policies -u ***** -o maf-holding -e dev -n DoesntExistYet -d . -b /testing-node-modules -p *****
2. Checked the Proxy in Apigee Edge and noticed that node_modules are not present
3. In Apigee Edge, I added and attached a random policy.
4. Went back to my terminal, and ran the same command in step 1 again.
5. Went back to apigee edge, saw that a new version of DoesntExistYet is deployed but noticed that node_modules are present this time.

Note: The random Policy that I added in step 3 was kept as intended.

[whitlockjc]

Any chance this is a duplicate of #131?

[classicalguss]

@whitlockjc Yeah I guess so. If --preserve-policies is present, then node_modules seems to be added in zip files.

And weirdly enough, I also noticed that both the old and new versions were deployed (and not just present) on dev environment.

[whitlockjc]

You shouldn't need to use --preserve-policies for deployhostedtarget, maybe that is the thing breaking everything. When you use deployhostedtarget, it only modifies the Hosted Target sources and would not affect your policies.

[classicalguss]

But it does actually remove the policies in my proxy when I don't include --preserver-policies

[whitlockjc]

Let me take a look. Maybe that's expected and this is a duplicate of #131.

[classicalguss]

My work around was to temporarily delete node_modules whenever I want to deploy and just run deployproxy

It seems to work perfectly without any issues. And after deploying, I run npm installagain locally to get back the node_modules locally. I'm not sure what's the purpose of adding node_modules in .zip, since my Proxy seems to work just fine without them.

[whitlockjc]

It's a relic of the old Trireme support that needed the modules uploaded as part of the proxy bundle. The only time we should bundle up anything is if you use the -b option to have us bundle up your bundled dependencies. I'm gonna close this and mark it a duplicate of #131. Please cc yourself there if you're interested in following along.