Respect proxy settings in postinstall

advmtue commented 1 year ago

Please ensure all items are complete before opening.

[x] Tick to sign-off your agreement to the Developer's Certificate of Origin
- Note: proxy-agent is licensed under MIT, which is compatible with this projects' Apache 2.0.
[ ] You have added tests for any code changes
- Note: postinstall.js does not have associated test files. I have not changed the structure of this code to make it testable, as that would require more work than introducing the wanted behaviour.
- I have, however, performed manual testing as described below.
[x] You have updated the CHANGES.md
[x] You have completed the PR template below:

What

Add support for the postinstall.js script to support proxy settings. Updated CHANGES.md and README.md.

The postinstall.js file was automatically formatted using the Eslint settings for the repository. This happens automatically for me in my editor. If you prefer I revert that styling I can look into it.

How

Use proxy-agent to configure the HTTP agent when making requests. proxy-agent respects the typical environment variables http_proxy, https_proxy and no_proxy.

Originally I went down the path of introducing this behaviour without a library. As it turns out, handling all the interactions for these environment variables is rather complex, so it seemed a better idea to not roll my own functionality.

Testing

Manual testing. The postinstall.js script was not in a state to support automated testing before introducing these changes.

Test type: no_proxy is not set.

Typically expecting to see that http_proxy has no effect, and https_proxy does.

Expected Behavior	As Expected?	no_proxy	http_proxy	HTTP_PROXY	https_proxy	HTTPS_PROXY
Pull from IBM	Yes	unset	unset	unset	unset	unset
Pull from IBM	Yes	unset	http://localhost:3128	unset	unset	unset
Pull from IBM	Yes	unset	unset	http://localhost:3128	unset	unset
Pull from Proxy	Yes	unset	unset	unset	http://localhost:3128	unset
Pull from Proxy	Yes	unset	unset	unset	unset	http://localhost:3128

Test type: no_proxy is set with config not containing IBM domain.

Typically expecting to see that http_proxy has no effect, and https_proxy does.

Expected Behavior	As Expected?	no_proxy	http_proxy	HTTP_PROXY	https_proxy	HTTPS_PROXY
Pull from IBM	Yes	`google.com,yahoo.com`	http://localhost:3128	unset	unset	unset
Pull from IBM	Yes	`google.com,yahoo.com`	unset	http://localhost:3128	unset	unset
Pull from Proxy	Yes	`google.com,yahoo.com`	unset	unset	http://localhost:3128	unset
Pull from Proxy	Yes	`google.com,yahoo.com`	unset	unset	unset	http://localhost:3128

Test type: no_proxy is set with config containing IBM domain.

Typically expecting to see that http_proxy and https_proxy have no effect as they are overridden by no_proxy.

Expected Behavior	As Expected?	no_proxy	http_proxy	HTTP_PROXY	https_proxy	HTTPS_PROXY
Pull from IBM	Yes	`google.com,*.ibm.com`	http://localhost:3128	unset	unset	unset
Pull from IBM	Yes	`google.com,*.ibm.com`	unset	http://localhost:3128	unset	unset
Pull from IBM	Yes	`google.com,*.ibm.com`	unset	unset	http://localhost:3128	unset
Pull from IBM	Yes	`google.com,*.ibm.com`	unset	unset	unset	http://localhost:3128

Test Logs

Check the proxy-agent output lines to see whether the proxy is configured for each command.

Logs

[adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +6ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent HTTP_PROXY=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent http_proxy=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent HTTPS_PROXY=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Request URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms proxy-agent Proxy URL: 'http://localhost:3128' +0ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent https_proxy=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Request URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +6ms proxy-agent Proxy URL: 'http://localhost:3128' +0ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent no_proxy="google.com,yahoo.com" HTTP_PROXY=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent no_proxy="google.com,yahoo.com" http_proxy=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent no_proxy="google.com,yahoo.com" HTTPS_PROXY=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Request URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms proxy-agent Proxy URL: 'http://localhost:3128' +1ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent no_proxy="google.com,yahoo.com" https_proxy=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Request URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms proxy-agent Proxy URL: 'http://localhost:3128' +0ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent no_proxy="google.com,*.ibm.com" HTTP_PROXY=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent no_proxy="google.com,*.ibm.com" http_proxy=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent no_proxy="google.com,*.ibm.com" HTTPS_PROXY=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms --- [adamtuechler] ~/src/advmtue/mq-mqi-nodejs % DEBUG=proxy-agent no_proxy="google.com,*.ibm.com" https_proxy=http://localhost:3128 npm run postinstall > ibmmq@2.0.1 postinstall > node postinstall.js Downloading IBM MQ Redistributable C Client runtime libraries - version 9.3.3.0 Getting https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz proxy-agent Creating new ProxyAgent instance: undefined +0ms proxy-agent Proxy not enabled for URL: 'https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/9.3.3.0-IBM-MQC-Redist-LinuxX64.tar.gz' +5ms

Issues

postinstall.js fails while running behind proxy #65

ibmmqmet commented 1 year ago

Thanks for this.

Coincidentally I'd also made the eshint changes in my master branch. But other changes I've got pending made it a bit harder to merge automatically. I've now managed that merge, and the PR should show up as accepted next time I do a proper release - currently waiting on some other base MQ changes to be public before I do that.

advmtue commented 11 months ago

Hiya @ibmmqmet, is there a timeline when this could be merged?

ibmmqmet commented 11 months ago

I've got an update planned to coincide with the next MQ CD release, as there are some other functional enhancements that will go in at that time. This will get merged then. Though I can't give a date for it publicly.

ibm-messaging / mq-mqi-nodejs