Closed ninrod closed 6 years ago
kylecordes
commented
6 years ago That is a very interesting firewall restriction. Numerous benign public websites are hosted/accelerated on Cloudflare... is it feasible to ask a platform like Angular to ensure that every asset that will be fetched by anything in its toolchain, will be hosted somewhere other than Cloudflare?
hansl
commented
6 years ago We use whatever NPM proxy you have setup. One of the thing that changed in 6.1 was to use the auth configuration you have for NPM. Could you verify your .npmrc file for proxy or authentication methods?
ninrod
commented
6 years ago Hi @hansl, I do not think I have anything auth or proxy related in my npmrc.
I just point to my artifactory npm registry which mirrors the global oficial npm registry.
Here's the full output of npm config list (I'm root because I'm inside a docker container):
~ ➜ npm config list
; cli configs
metrics-registry = "http://artifactory/artifactory/api/npm/npm"
scope = ""
user-agent = "npm/5.6.0 node/v8.11.3 linux x64"
; globalconfig /usr/local/etc/npmrc
disturl = "http://artifactory/artifactory/nodejs-dist"
registry = "http://artifactory/artifactory/api/npm/npm"
; node bin location = /usr/local/bin/node
; cwd = /root
; HOME = /root
; "npm config ls -l" to show all defaults.
~ ➜ if this was working with 6.0.8, am I supposed to have to configure anything different for 6.1.0?
I think that there's a great chance users which are behind a firewall are going to hit this.
hansl
commented
6 years ago Could you try to dig artifactory or find out what IP address it is at =?
ninrod
commented
6 years ago artifactory is just an internal ip inside our intranet. 172.17.xxx.xxx.
ninrod
commented
6 years ago Hi @hansl. I have gathered more data and now I'm almost 100% sure that this is a critical bug. Can you verify?
I've run a tcpdump session in both 6.0.8 and in 6.1.0 while issuing ng update. here are the facts:
in 6.0.8 ng update did not try to connect to a cloudflare server. not once. not even a single packet. it connects to artifactory. I suppose that is the logical expected behaviour.
Now in 6.1.0 ng updatetries to connect to a cloudflare server and not to artifactory, which is the configured registry in the global npmrc in my machine. there is not one single tcp packet routed to artifactory in the entire session.
here is a sample snippet from 6.1.0 captured with help from tcpdump -w eth0.pcap -i eht0 (workbench is my machine):
18:20:33.982013 IP workbench.39916 > 104.16.18.35.https: Flags [S], seq 1010461359, win 29200, options [mss 1460,sackOK,TS val 4342263 ecr 0,nop,wscale 7], length 0
18:20:33.982557 IP 192.168.65.1.domain > workbench.51048: 39844 12/0/0 A 104.16.18.35, A 104.16.20.35, A 104.16.21.35, A 104.16.27.35, A 104.16.23.35, A 104.16.19.35, A 104.16.25.35, A 104.16.16.35, A 104.16.26.35, A 104.16.24.35, A 104.16.17.35, A 104.16.22.35 (228)
18:20:33.983573 IP workbench.39918 > 104.16.18.35.https: Flags [S], seq 3132673246, win 29200, options [mss 1460,sackOK,TS val 4342263 ecr 0,nop,wscale 7], length 0
18:20:33.983856 IP 192.168.65.1.domain > workbench.33715: 16090 12/0/0 A 104.16.18.35, A 104.16.20.35, A 104.16.21.35, A 104.16.27.35, A 104.16.23.35, A 104.16.19.35, A 104.16.25.35, A 104.16.16.35, A 104.16.26.35, A 104.16.24.35, A 104.16.17.35, A 104.16.22.35 (228)
18:20:33.984381 IP workbench.34588 > 192.168.65.1.domain: 44828+ A? registry.npmjs.org. (36)
18:20:33.984970 IP 192.168.65.1.domain > workbench.38433: 65065 12/0/0 A 104.16.18.35, A 104.16.20.35, A 104.16.21.35, A 104.16.27.35, A 104.16.23.35, A 104.16.19.35, A 104.16.25.35, A 104.16.16.35, A 104.16.26.35, A 104.16.24.35, A 104.16.17.35, A 104.16.22.35 (228)
18:20:33.985043 IP 192.168.65.1.domain > workbench.58586: 52881 12/0/0 A 104.16.18.35, A 104.16.20.35, A 104.16.21.35, A 104.16.27.35, A 104.16.23.35, A 104.16.19.35, A 104.16.25.35, A 104.16.16.35, A 104.16.26.35, A 104.16.24.35, A 104.16.17.35, A 104.16.22.35 (228)
18:20:33.986612 IP workbench.42800 > 192.168.65.1.domain: 37758+ A? registry.npmjs.org. (36)note: this 104.16.18.35 ip is a cloudflare server.
Now here is a sample tcpdump snippet from 6.0.8:
note: sbcdf581.bcnet.bcb.gov.br is the server address of our internal artifactory server
18:30:06.192556 IP workbench.43058 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:235, ack 1, win 229, length 234: HTTP: GET /artifactory/api/npm/npm/codelyzer HTTP/1.1
18:30:06.192919 IP workbench.43060 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:235, ack 1, win 229, length 234: HTTP: GET /artifactory/api/npm/npm/bootstrap HTTP/1.1
18:30:06.194848 IP workbench.43062 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:238, ack 1, win 229, length 237: HTTP: GET /artifactory/api/npm/npm/concurrently HTTP/1.1
18:30:06.197632 IP workbench.43064 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:242, ack 1, win 229, length 241: HTTP: GET /artifactory/api/npm/npm/enhanced-resolve HTTP/1.1
18:30:06.206979 IP workbench.43066 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:233, ack 1, win 229, length 232: HTTP: GET /artifactory/api/npm/npm/jasmine HTTP/1.1
18:30:06.209231 IP workbench.43068 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:238, ack 1, win 229, length 237: HTTP: GET /artifactory/api/npm/npm/jasmine-core HTTP/1.1
18:30:06.209311 IP workbench.43070 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:247, ack 1, win 229, length 246: HTTP: GET /artifactory/api/npm/npm/jasmine-spec-reporter HTTP/1.1
18:30:06.215753 IP workbench.43072 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:237, ack 1, win 229, length 236: HTTP: GET /artifactory/api/npm/npm/json-server HTTP/1.1
18:30:06.220899 IP workbench.43074 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:231, ack 1, win 229, length 230: HTTP: GET /artifactory/api/npm/npm/karma HTTP/1.1
18:30:06.224766 IP workbench.43076 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:247, ack 1, win 229, length 246: HTTP: GET /artifactory/api/npm/npm/karma-chrome-launcher HTTP/1.1
18:30:06.227195 IP workbench.43078 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:235, ack 1, win 229, length 234: HTTP: GET /artifactory/api/npm/npm/karma-cli HTTP/1.1
18:30:06.233933 IP workbench.43080 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:258, ack 1, win 229, length 257: HTTP: GET /artifactory/api/npm/npm/karma-coverage-istanbul-reporter HTTP/1.1
18:30:06.242451 IP workbench.43084 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:248, ack 1, win 229, length 247: HTTP: GET /artifactory/api/npm/npm/karma-firefox-launcher HTTP/1.1
18:30:06.242538 IP workbench.43082 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:239, ack 1, win 229, length 238: HTTP: GET /artifactory/api/npm/npm/karma-jasmine HTTP/1.1
18:30:06.246609 IP workbench.43086 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:253, ack 1, win 229, length 252: HTTP: GET /artifactory/api/npm/npm/karma-jasmine-html-reporter HTTP/1.1
18:30:06.254574 IP workbench.43088 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:243, ack 1, win 229, length 242: HTTP: GET /artifactory/api/npm/npm/npm-check-updates HTTP/1.1
18:30:06.255544 IP workbench.43090 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:237, ack 1, win 229, length 236: HTTP: GET /artifactory/api/npm/npm/npm-run-all HTTP/1.1
18:30:06.261678 IP workbench.43094 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:236, ack 1, win 229, length 235: HTTP: GET /artifactory/api/npm/npm/protractor HTTP/1.1
18:30:06.262052 IP workbench.43092 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:265, ack 1, win 229, length 264: HTTP: GET /artifactory/api/npm/npm/protractor-jasmine2-screenshot-reporter HTTP/1.1
18:30:06.270928 IP workbench.43096 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:232, ack 1, win 229, length 231: HTTP: GET /artifactory/api/npm/npm/tslint HTTP/1.1
18:30:06.273485 IP workbench.43098 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:233, ack 1, win 229, length 232: HTTP: GET /artifactory/api/npm/npm/ts-node HTTP/1.1
18:30:06.279835 IP workbench.43100 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:236, ack 1, win 229, length 235: HTTP: GET /artifactory/api/npm/npm/typescript HTTP/1.1
18:30:06.280514 IP workbench.43102 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:233, ack 1, win 229, length 232: HTTP: GET /artifactory/api/npm/npm/webpack HTTP/1.1
18:30:06.293248 IP workbench.43106 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:233, ack 1, win 229, length 232: HTTP: GET /artifactory/api/npm/npm/core-js HTTP/1.1
18:30:06.293362 IP workbench.43104 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:238, ack 1, win 229, length 237: HTTP: GET /artifactory/api/npm/npm/classlist.js HTTP/1.1
18:30:06.300966 IP workbench.43110 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:244, ack 1, win 229, length 243: HTTP: GET /artifactory/api/npm/npm/ngx-feature-toggle HTTP/1.1
18:30:06.301088 IP workbench.43108 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:236, ack 1, win 229, length 235: HTTP: GET /artifactory/api/npm/npm/ng-snotify HTTP/1.1
18:30:06.312663 IP workbench.43112 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:230, ack 1, win 229, length 229: HTTP: GET /artifactory/api/npm/npm/rxjs HTTP/1.1
18:30:06.312722 IP workbench.43116 > sbcdf581.bcnet.bcb.gov.br.http: Flags [P.], seq 1:241, ack 1, win 229, length 240: HTTP: GET /artifactory/api/npm/npm/ngx-permissions HTTP/1.1ng update needs to call a cloudflare server to update projects? 6.0.8 to 6.1.0 where my configured registry (artifactory) is not getting called anymore by the cli? hansl
commented
6 years ago what changed? Is this really needed? Is this working as intended?
Doesn't look like it.
why do ng update needs to call a cloudflare server to update projects?
Shouldn't need to.
why the sudden change in behaviour from 6.0.8 to 6.1.0 where my configured registry (artifactory) is not getting called anymore by the cli?
No idea, need to look.
If this is intended, can I opt out?
We need to investigate first.
can I setup an internal artifactory/binrepo to server what the cloudflare is serving?
We don't know what cloudflare is serving. I have to look it up.
I will investigate. We should not make any request to non npmjs servers. AFAIK the only feature added to ng update was the auth stuff we talked above. I'll do a diff between 6.0.8 and 6.1.0 to see what could have changed.
hansl
commented
6 years ago Okay after looking a big deeper into this, it seems we always contact registry.npmjs.org now, which resolves to the address above:
🏢 dig registry.npmjs.org
; <<>> DiG 9.10.6 <<>> registry.npmjs.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52664
;; flags: qr rd ra; QUERY: 1, ANSWER: 12, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;registry.npmjs.org. IN A
;; ANSWER SECTION:
registry.npmjs.org. 255 IN A 104.16.23.35
registry.npmjs.org. 255 IN A 104.16.21.35
registry.npmjs.org. 255 IN A 104.16.25.35
registry.npmjs.org. 255 IN A 104.16.17.35
registry.npmjs.org. 255 IN A 104.16.18.35
registry.npmjs.org. 255 IN A 104.16.24.35
registry.npmjs.org. 255 IN A 104.16.19.35
registry.npmjs.org. 255 IN A 104.16.22.35
registry.npmjs.org. 255 IN A 104.16.27.35
registry.npmjs.org. 255 IN A 104.16.20.35
registry.npmjs.org. 255 IN A 104.16.26.35
registry.npmjs.org. 255 IN A 104.16.16.35
;; Query time: 63 msec
;; SERVER: 172.16.255.1#53(172.16.255.1)
;; WHEN: Thu Jul 26 15:14:06 PDT 2018
;; MSG SIZE rcvd: 239I wiretapped my process and could not find any contact outside of one of those.
Now, why we always contact the registry is unknown and I'll look at it, but this is not a security risk and I'll downgrade the priority from critical.
ninrod
commented
6 years ago hi @hansl, I've come up with even more data and now I'm pretty sure what the issue is. Can you see if my understanding is correct?
I've fired up wireshark using 6.0.8 ng update and now 6.1.1 ng update.
My conclusion is this:
the cloudflare servers are really serving the npm registry. So in the context of this issue, cloudflare is registry.npmjs.org.
6.0.8 correctly picked up the registry in the npm configuration if the user configured an alternative registry, which is my case.
In 6.1.0 and 6.1.1 the cli is ignoring the alternative registry set by the user. In my case:
$ npm config set registry http://artifactory/artifactory/api/npm/npm --globalSo based on this, I'd say it remains critical.
I reached this conclusion looking at these dns packets which where captured on ng update in 6.0.8 and 6.1.1:
here is the result of 6.0.8:
now here is the result of 6.1.1:
note: my npm configuration remained intact throughout the captures.
hansl
commented
6 years ago Okay I found the culprit. Setting the global registry when you're using nvm isn't setting the same file, and rc finds the default one.
There is a lack of information in this PR and I'll try to check with Charles why that PR was done.
dominikg
commented
6 years ago the cloudflare stuff for registry.npmjs.org is expected, this was added by npm inc a while ago: https://twitter.com/ceejbot/status/1000147930362204160
ninrod
commented
6 years ago @dominikg issue here is not the cloudflare stuff, but the fact that the cli is ignoring the alternative registry set by the user. In our case, we mirror the npm registry through an internal artifactory server located on premises.
ninrod
commented
6 years ago @hansl, I think I've found the root cause:
here we can see that rc does not look at /usr/local/etc/npmrc which is the file set if we issue npm config set --global commands.
so I'm concluding that not only the registry but any configuration set by the user globally will be completely ignored.
angular-automatic-lock-bot[bot]
commented
5 years ago This issue has been automatically locked due to inactivity. Please file a new issue if you are encountering a similar or related problem.
Read more about our automatic conversation locking policy.
This action has been performed automatically by a bot.
Bug Report or Feature Request (mark with an
x)Command (mark with an
x)Versions
Repro steps
npm config set registry http://yourregistryThe log given by the failure
Desired functionality
6.0.8and below.Mention any other details that might be useful
n/a
Hi, I use angular behind a corp firewall and after upgrading to 6.1.0 cli with
ng updateI'm seing that the cli is trying to connect to a cloudflare ip:this was not happening prior to 6.1.0. What is the cli trying to do with cloudflare? is this really necessary?
I had to downgrade to 6.0.8, run
ng update @angular/coreand thenng update @angular/cli. after the last step,ng updatenow times out on me because I do not have access to104.16.18.35which is a cloudflare server.