search

cnpm / cnpmjs.org

‼️ ‼️ ‼️ ‼️ DEPRECATED, please use https://github.com/cnpm/cnpmcore ‼️ ‼️ ‼️ ‼️
https://npmmirror.com
Other
3.55k stars 746 forks source link

Request timeouts almost every day #909

Closed glen-84 closed 8 years ago

glen-84 commented 8 years ago

Almost every day we get the following:

Sync packages from official registry failed.
Start sync time is Mon May 02 2016 10:06:38 GMT+0000 (UTC).
Error message is ResponseTimeoutError: Request timeout for 300000ms, GET https://registry.npm.taobao.org/-/short 200 (connected: true, keepalive socket: false, agent status: {"createSocketCount":4,"closeSocketCount":4,"errorSocketCount":0,"timeoutSocketCount":0,"requestCount":3,"freeSockets":{},"sockets":{},"requests":{}})
headers: {"date":"Mon, 02 May 2016 10:02:06 GMT","content-type":"application/json; charset=utf-8","content-length":"4711183","connection":"keep-alive","etag":"\"47e30f-sAIl+yDS3z2Vzu6oJL0izQ\"","x-readtime":"2992"} (sync package error)
ResponseTimeoutError: Request timeout for 300000ms, GET https://registry.npm.taobao.org/-/short 200 (connected: true, keepalive socket: false, agent status: {"createSocketCount":4,"closeSocketCount":4,"errorSocketCount":0,"timeoutSocketCount":0,"requestCount":3,"freeSockets":{},"sockets":{},"requests":{}})
headers: {"date":"Mon, 02 May 2016 10:02:06 GMT","content-type":"application/json; charset=utf-8","content-length":"4711183","connection":"keep-alive","etag":"\"47e30f-sAIl+yDS3z2Vzu6oJL0izQ\"","x-readtime":"2992"} (sync package error)
at Timeout._onTimeout (/con/home/cnpm/node_modules/urllib/lib/urllib.js:647:13)
at tryOnTimeout (timers.js:224:11)
at Timer.listOnTimeout (timers.js:198:5).

Today we got:

Sync packages from official registry failed.
Start sync time is Wed May 04 2016 10:05:26 GMT+0000 (UTC).
Error message is JSONResponseFormatError: Unexpected end of input (data json format: "[\"0\",\"0.0\",\"0.workspace\",\"00\",\"001\",\"001_skt\",\"001_test\",\"003\",\"007\",\"008-somepackage\",\"009\",\"00k-node\",\"01\",\"01-simple\",\"0126af95c0e2d9b0a7c78738c4c00a860b04acc8\",\"03\",\"04uppercase\",\"06_byvoidmodule\",\"06_cli_applications_using_nodejs\",\"0815\",\"09aac0b7d986b720253a65a47d1fd3d9\",\"0card-images-helper\",\"0flux\",\"0latency\",\"0model\",\"0s\",\"0x\",\"0x21\",\"0x23\",\"0xff\",\"1\",\"1-1-help-desk-system\",\"1-click-bom\",\"1-liners\",\"1-wire-js\",\"1.0.1\",\"1.0.2\",\"10\",\"100\",\"100-prisoners\",\"101\",\"101-es6\",\"101-tomekwi\",\"10119-dep\",\"101" ...skip... "k\",\"azuma-test-githook\",\"azumio\",\"azuqua\",\"azura\",\"azure\",\"azure-ad-jwt\",\"azure-ad-jwt-cache\",\"azure-ad-jwt-mod\",\"azure-angular-providers\",\"azure-api\",\"azure-arm-apiapp\",\"azure-arm-authorization\",\"azure-arm-batch\",\"azure-arm-cdn\",\"azure-arm-commerce\",\"azure-arm-compute\",\"azure-arm-datalake-analytics\",\"azure-arm-datalake-store\",\"azure-arm-dns\",\"azure-arm-hdinsight\",\"azure-arm-hdinsight-jobs\",\"azure-arm-insights\",\"azure-arm-intune\",\"azure-arm-keyvault\",\"azure-arm-network\",\"azure-arm-rediscache\",\"azure-arm-res"), GET https://registry.npm.taobao.org/-/short 200 (connected: true, keepalive socket: false, agent status: {"createSocketCount":6,"closeSocketCount":6,"errorSocketCount":0,"timeoutSocketCount":0,"requestCount":5,"freeSockets":{},"sockets":{},"requests":{}})
headers: {"date":"Wed, 04 May 2016 10:02:07 GMT","content-type":"application/json; charset=utf-8","content-length":"4727619","connection":"keep-alive","etag":"\"482343-iBRdOfOAKh31CUDt8vPatg\"","x-readtime":"2597"} (sync package error)
JSONResponseFormatError: Unexpected end of input (data json format: "[\"0\",\"0.0\",\"0.workspace\",\"00\",\"001\",\"001_skt\",\"001_test\",\"003\",\"007\",\"008-somepackage\",\"009\",\"00k-node\",\"01\",\"01-simple\",\"0126af95c0e2d9b0a7c78738c4c00a860b04acc8\",\"03\",\"04uppercase\",\"06_byvoidmodule\",\"06_cli_applications_using_nodejs\",\"0815\",\"09aac0b7d986b720253a65a47d1fd3d9\",\"0card-images-helper\",\"0flux\",\"0latency\",\"0model\",\"0s\",\"0x\",\"0x21\",\"0x23\",\"0xff\",\"1\",\"1-1-help-desk-system\",\"1-click-bom\",\"1-liners\",\"1-wire-js\",\"1.0.1\",\"1.0.2\",\"10\",\"100\",\"100-prisoners\",\"101\",\"101-es6\",\"101-tomekwi\",\"10119-dep\",\"101" ...skip... "k\",\"azuma-test-githook\",\"azumio\",\"azuqua\",\"azura\",\"azure\",\"azure-ad-jwt\",\"azure-ad-jwt-cache\",\"azure-ad-jwt-mod\",\"azure-angular-providers\",\"azure-api\",\"azure-arm-apiapp\",\"azure-arm-authorization\",\"azure-arm-batch\",\"azure-arm-cdn\",\"azure-arm-commerce\",\"azure-arm-compute\",\"azure-arm-datalake-analytics\",\"azure-arm-datalake-store\",\"azure-arm-dns\",\"azure-arm-hdinsight\",\"azure-arm-hdinsight-jobs\",\"azure-arm-insights\",\"azure-arm-intune\",\"azure-arm-keyvault\",\"azure-arm-network\",\"azure-arm-rediscache\",\"azure-arm-res"), GET https://registry.npm.taobao.org/-/short 200 (connected: true, keepalive socket: false, agent status: {"createSocketCount":6,"closeSocketCount":6,"errorSocketCount":0,"timeoutSocketCount":0,"requestCount":5,"freeSockets":{},"sockets":{},"requests":{}})
headers: {"date":"Wed, 04 May 2016 10:02:07 GMT","content-type":"application/json; charset=utf-8","content-length":"4727619","connection":"keep-alive","etag":"\"482343-iBRdOfOAKh31CUDt8vPatg\"","x-readtime":"2597"} (sync package error)
at Object.parse (native)
at parseJSON (/con/home/cnpm/node_modules/urllib/lib/urllib.js:733:24)
at /con/home/cnpm/node_modules/urllib/lib/urllib.js:606:23
at decodeContent (/con/home/cnpm/node_modules/urllib/lib/urllib.js:471:14)
at IncomingMessage. (/con/home/cnpm/node_modules/urllib/lib/urllib.js:587:7)
at emitNone (events.js:85:20)
at IncomingMessage.emit (events.js:179:7)
at endReadableNT (_stream_readable.js:913:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickCallback (internal/process/next_tick.js:98:9).

Is there any way to improve the connection? Our server is in the US.

dead-horse commented 8 years ago

https://github.com/cnpm/cnpmjs.org/blob/master/config/index.js#L182

you can change the source registry to http://registry.cnpmjs.org , this server is in US

drsect0r commented 8 years ago

@dead-horse Is there maybe some load balancing doing some magic? I did a traceroute from Europe (Malta) and the traffic seems to be routed to/trough Singapore.

mtr --report-wide registry.cnpmjs.org
                                                     Loss%   Snt   Last   Avg  Best  Wrst StDev
...
  6.|-- ae-0-11.bar1.Milan1.Level3.net                0.0%    10   38.8  37.8  35.3  42.6   2.5
  7.|-- 4.68.71.58   
  8.|-- if-ae-1-2.tcore1.FR0-Frankfurt.as6453.net     0.0%    10  288.0 290.6 287.4 293.2   1.8
  9.|-- if-ae-4-2.tcore2.FNM-Frankfurt.as6453.net     0.0%    10  302.3 304.5 301.3 313.6   3.7
 10.|-- if-ae-9-2.tcore1.PVU-Paris.as6453.net         0.0%    10  293.8 293.3 291.3 296.0   1.7
 11.|-- if-ae-2-2.tcore1.PYE-Paris.as6453.net         0.0%    10  299.6 300.0 297.8 303.5   1.8
 12.|-- if-ae-8-1600.tcore1.WYN-Marseille.as6453.net  0.0%    10  297.4 294.4 292.5 297.8   1.8
 13.|-- if-ae-9-6.tcore1.MLV-Mumbai.as6453.net        0.0%    10  288.2 286.9 281.9 292.7   3.8
 14.|-- if-ae-2-2.tcore2.MLV-Mumbai.as6453.net        0.0%    10  291.1 289.2 287.8 291.1   0.9
 15.|-- if-ae-16-2.tcore1.SVW-Singapore.as6453.net    0.0%    10  298.3 295.6 293.4 298.3   1.7
 16.|-- 180.87.12.50                                  0.0%    10  303.0 304.1 302.8 306.2   0.9
 17.|-- 116.251.66.25                                 0.0%    10  296.1 297.1 294.4 301.7   2.3
 18.|-- ???                                          100.0    10    0.0   0.0   0.0   0.0   0.0
 19.|-- 47.88.189.193                                 0.0%    10  291.6 293.8 291.6 297.3   1.8
dead-horse commented 8 years ago

sorry, maybe registry.cnpmjs.org is migrated to Singapore by @fengmk2 ?

If registry.cnpmjs.org is still timeout, you can set sourceRegistry to https://registry.npmjs.com, with tow points to notice about:

  1. turn sourceNpmRegistryIsCNpm off. https://github.com/cnpm/cnpmjs.org/blob/master/config/index.js#L186
  2. make sure the first time must synchronize from cnpm, because npm's registry no logger provide the API we needed.
fengmk2 commented 8 years ago

registry.cnpmjs.org is on Singapore now.

glen-84 commented 8 years ago

We haven't seen any timeouts yet from the Singapore server.

Thanks for your help. (=