search

npm / registry-issue-archive

An archive of the old npm registry issue tracker
https://npm.community
249 stars 47 forks source link

'publish' hook fired before package available in registry #126

Open jesstelford opened 7 years ago

jesstelford commented 7 years ago

This just happened (02:30UTC, Wednesday, 15 February 2017)

  1. Performed an npm publish, which succeeded.
  2. Then correctly received the publish event from npm to our API
  3. Our server immediately used the npm cli to try and install the package
  4. The npm cli errored with:
Command failed: /bin/sh -c npm i @domain-group/fe-pa-allhomes-agent-profile@5.2.0
npm ERR! Linux 4.4.39-34.54.amzn1.x86_64
npm ERR! argv "/root/.nvm/versions/node/v6.6.0/bin/node" "/root/.nvm/versions/node/v6.6.0/bin/npm" "i" "@domain-group/fe-pa-allhomes-agent-profile@5.2.0"
npm ERR! node v6.6.0
npm ERR! npm  v3.10.3

npm ERR! No compatible version found: @domain-group/fe-pa-allhomes-agent-profile@5.2.0
npm ERR! Valid install targets:
npm ERR! 5.1.0, 5.0.0, 4.1.4, 4.1.3, 4.1.2, 4.1.1, 4.1.0, 4.0.0, 3.1.0, 3.0.0, 2.1.0, 2.0.1, 2.0.0, 1.0.2, 1.0.1, 1.0.0

(unfortuantely the box is ephemeral, so I don't have the .log file to share)

Retrying the installation just moments later worked successfully.

My guess is the hook was fired before the package was replicated to all data stores globally (our server is located in the Sydney, Australia AWS region), so when we tried to install, it wasn't quite ready yet.

Update: See https://github.com/npm/registry/issues/126#issuecomment-293431336 for a workaround by @soldair

soldair commented 7 years ago

i see how this could happen. sydney package metadata is served from aws sydney package events are triggered in us-west which is following changes in west.

i'll have to ponder how we go about solving this issue. the only helpful suggestion i have at the moment is adding a wait :fearful:

jesstelford commented 7 years ago

Alright, thanks @soldair! Any suggested wait time? Do you know how long approximately it takes to propagate around the world?

FYI, it just now (00:46UTC 16 February 2017) happened again (odd that it should happen twice so close together, but not in the last 4-5 months):

Command failed: /bin/sh -c npm i @domain-group/fe-pa-allhomes-agency-profile@2.3.3
npm ERR! Linux 4.4.39-34.54.amzn1.x86_64
npm ERR! argv "/root/.nvm/versions/node/v6.6.0/bin/node" "/root/.nvm/versions/node/v6.6.0/bin/npm" "i" "@domain-group/fe-pa-allhomes-agency-profile@2.3.3"
npm ERR! node v6.6.0
npm ERR! npm  v3.10.3

npm ERR! No compatible version found: @domain-group/fe-pa-allhomes-agency-profile@2.3.3
npm ERR! Valid install targets:
npm ERR! 2.3.2, 2.3.1, 2.3.0, 2.2.1, 2.2.0, 2.1.1, 2.1.0, 2.0.4, 2.0.3, 2.0.2, 2.0.1, 2.0.0, 1.4.0, 1.3.3, 1.3.2, 1.3.1, 1.3.0, 1.2.8, 1.2.7, 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0, 1.1.6, 1.1.5, 1.1.4, 1.1.3, 1.1.2, 1.1.1, 1.1.0, 1.0.0, 0.1.0

For clarity, I should point out that the two packages are different. The first is ...agent..., and this is ...agency...

jesstelford commented 7 years ago

Happened again to us today:

Command failed: /bin/sh -c npm i @domain-group/fe-co-myd-widgets@1.3.1
npm ERR! Linux 4.4.39-34.54.amzn1.x86_64
npm ERR! argv "/root/.nvm/versions/node/v6.6.0/bin/node" "/root/.nvm/versions/node/v6.6.0/bin/npm" "i" "@domain-group/fe-co-myd-widgets@1.3.1"
npm ERR! node v6.6.0
npm ERR! npm  v3.10.3

npm ERR! No compatible version found: @domain-group/fe-co-myd-widgets@1.3.1
npm ERR! Valid install targets:
npm ERR! 1.3.0, 1.2.0, 1.1.1, 1.1.0, 1.0.1, 1.0.0

:(

jesstelford commented 7 years ago

@soldair A quick update: This has happened a further 6 times since my last post above (all on different packages).

We haven't yet added a wait into our hook.

soldair commented 7 years ago

sadly the only reliable way right now is to poll the package endpoint in question (https://registry.npmjs.org/mypackage) until the modified time is >= to the modified time in the document or the version in question exists. this could be wrapped in a module =/

long term we need the delivery of webhooks to a region be based on the progress of the replicatipon feed for the region.

soldair commented 7 years ago

@jesstelford we have a plan to fix this but its not complete yet. :(

i made this module. https://www.npmjs.com/package/wait-for-package-replication

if you run this in the hook receiver before triggering your work it'll make sure to block till things have made it to your region.

jesstelford commented 7 years ago

Oh wow, that is excellent! Thanks @soldair - we're definitely going to make use of this 🎉