nice-registry / nice-package

📦 Clean up messy package metadata from the npm registry
35 stars 7 forks source link

nice-package assumes everything is a GitHub repo #10

Closed pdehaan closed 8 years ago

pdehaan commented 8 years ago

... which is a fair assumption, but I'm seeing some cases in my :poop::fire: https://github.com/pdehaan/npm-today repo where recently published modules may be gitlab or bitbucket or other random Git repos (or just random garbage).

Not sure if it's worth fixing, but a bit inconvenient when a user specifies a valid repo URL and after normalizing it through nice-package, it comes back null.

Example:

const fetchNicePackage = require('fetch-nice-package');

fetchNicePackage('conventional-gitlab-releaser')
  .then(({name, version, repository}) => ({name, version, repository}))
  .then(pkg => JSON.stringify(pkg, null, 2))
  .then(output => console.log(output))
  .catch(err => console.error(err));

Output:

{
  "name": "conventional-gitlab-releaser",
  "version": "1.0.14",
  "repository": null
}

Yet, it we look at http://npm.im/conventional-gitlab-releaser we can see npm sees the repository, and the package.json specifies a valid value at https://gitlab.com/hutson/conventional-gitlab-releaser/blob/master/package.json#L37-40

  "name": "conventional-gitlab-releaser",
  "repository": {
    "type": "git",
    "url": "https://gitlab.com/hutson/conventional-gitlab-releaser.git"
  },

Not sure if there is an easy way to have it return the original package.json repository.url (if there was one), if the parser wasn't able to parse the repository -- although that means that you'd potentially be getting back different keys depending on what repository people used.

pdehaan commented 8 years ago

Here's some really crude output from my test script. There are definitely a few unexpected results which I can't reproduce w/ fetch-nice-package so I think the https://registry.npmjs.org/-/all/static/today.json endpoint may be returning some funny data, or my code is most certainly buggy.

But you can see a bunch of invalid repos which don't deserve to parse, or a few bitbucket/gitlab repos. Plus a few extra random apache.org results.

Click to expand: [**aem-react-js**](http://npm.im/aem-react-js): - original: http://www.github.com/sinnerschrader/aem-react-js - parsed: null [**ajax-promise-es6**](http://npm.im/ajax-promise-es6): - original: github.com/dragfire/ajax-promise-es6.git - parsed: null [**arrowchat**](http://npm.im/arrowchat): - original: https://bitbucket.org/dwx/arrowchat-server - parsed: null [**backlog-js**](http://npm.im/backlog-js): - original: git://github.com/%3Anulab/backlog-js.git - parsed: null [**bot-languages**](http://npm.im/bot-languages): - original: git+ssh://git@gitlab.com/meowbot/bot-languages.git - parsed: null [**casket**](http://npm.im/casket): - original: github.com/derhuerst/casket - parsed: null [**cloudsim-grant**](http://npm.im/cloudsim-grant): - original: git+https://bitbucket.org/osrf/cloudsim-grant.git - parsed: null [**conventional-gitlab-releaser**](http://npm.im/conventional-gitlab-releaser): - original: git+https://gitlab.com/hutson/conventional-gitlab-releaser.git - parsed: null [**cordova**](http://npm.im/cordova): - original: https://git-wip-us.apache.org/repos/asf/cordova-cli.git - parsed: null [**cordova-android**](http://npm.im/cordova-android): - original: https://git-wip-us.apache.org/repos/asf/cordova-android.git - parsed: null [**cordova-ios**](http://npm.im/cordova-ios): - original: https://git-wip-us.apache.org/repos/asf/cordova-ios.git - parsed: null [**cordova-lib**](http://npm.im/cordova-lib): - original: git://git-wip-us.apache.org/repos/asf/cordova-lib.git - parsed: null [**dartboard**](http://npm.im/dartboard): - original: git+https://gitlab.com/bmcallis/dartboard.git - parsed: null [**dice-api**](http://npm.im/dice-api): - original: git://bitbucket.org/icecom/diceapi - parsed: null [**docs2md**](http://npm.im/docs2md): - original: https://github.com/aymericbeaumet/docs2md.git - parsed: {"type":"git","url":"https://github.com/aymericbeaumet/docs2md.git"} [**dummy-nodesample**](http://npm.im/dummy-nodesample): - original: https://sigurdvh.visualstudio.com/_git/NodeSample - parsed: null [**dummy-nodesample-2**](http://npm.im/dummy-nodesample-2): - original: https://sigurdvh.visualstudio.com/_git/NodeSample - parsed: null [**employeeapp**](http://npm.im/employeeapp): - original: Abdurraheem786@gmail.com - parsed: {"type":"git","url":"Abdurraheem786@gmail.com"} [**evand-iconfont**](http://npm.im/evand-iconfont): - original: git+https://gitlab.com/Eventfa/evand-icons.git - parsed: null [**flux-json-to-three**](http://npm.im/flux-json-to-three): - original: git+ssh://git@bitbucket.org/vannevartech/flux-json-to-three.git - parsed: null [**fs-advanced**](http://npm.im/fs-advanced): - original: https://www.github.com/KlwntSingh/fs-advanced - parsed: null [**general_fetch**](http://npm.im/general_fetch): - original: http://git.code.oa.com/halwu/general_fetch.git - parsed: null [**generator-python-lib**](http://npm.im/generator-python-lib): - original: git+https://gitlab.com/hutson/generator-python-lib.git - parsed: null [**gk-test-dep**](http://npm.im/gk-test-dep): - original: git+https://github.com/boennemann/gk-test-dep.git - parsed: {"type":"git","url":"git+https://github.com/boennemann/gk-test-dep.git"} [**grunt-sprigganjs-aseprite**](http://npm.im/grunt-sprigganjs-aseprite): - original: git+https://bitbucket.org/SUNRUSE/sprigganjs.git - parsed: null [**gsb-s**](http://npm.im/gsb-s): - original: ddc - parsed: null [**handsontable-helpers**](http://npm.im/handsontable-helpers): - original: git+https://github.com/artbels/handsontable-helpers.git - parsed: {"type":"git","url":"git+https://github.com/artbels/handsontable-helpers.git"} [**headers2md**](http://npm.im/headers2md): - original: https://github.com/aymericbeaumet/headers2md.git - parsed: {"type":"git","url":"https://github.com/aymericbeaumet/headers2md.git"} [**hubot-ibmcloud-utils**](http://npm.im/hubot-ibmcloud-utils): - original: github.com/ibm-cloud-solutions/hubot-ibmcloud-utils - parsed: null [**izy-server**](http://npm.im/izy-server): - original: https://github.com/izy-server/ - parsed: null [**js-kernel**](http://npm.im/js-kernel): - original: git+https://github.com/alexisvincent.io/js-kernel.git - parsed: null [**laup**](http://npm.im/laup): - original: no - parsed: null [**loopback-jwt-simple**](http://npm.im/loopback-jwt-simple): - original: git+https://github.com/tonghuiquanqiu/loopback-jwt-simple.git - parsed: {"type":"git","url":"git+https://github.com/tonghuiquanqiu/loopback-jwt-simple.git"} [**master-css**](http://npm.im/master-css): - original: git+https://wk-ux-team@bitbucket.org/wk-ux-team/master-css.git - parsed: null [**materia-server**](http://npm.im/materia-server): - original: git+ssh://git@github.com/webshell/materia-server.git - parsed: {"type":"git","url":"git+ssh://git@github.com/webshell/materia-server.git"} [**mdlib-dbs**](http://npm.im/mdlib-dbs): - original: git+https://gitlab.com/dsuess/mdlib-dbs.git - parsed: null [**mdlib-logger**](http://npm.im/mdlib-logger): - original: git+https://gitlab.com/dsuess/mdlib-logger.git - parsed: null [**minecraft-launcher**](http://npm.im/minecraft-launcher): - original: https://coding.net/u/ncbql/p/minecraft-launcher/git - parsed: null [**mongo-dynamic-indexer**](http://npm.im/mongo-dynamic-indexer): - original: git+https://bitbucket.org/sensibill/mongo-dynamic-indexer.git - parsed: null [**mora-bot-data**](http://npm.im/mora-bot-data): - original: git+https://github.com/program-bot/mora-bot-data.git - parsed: {"type":"git","url":"git+https://github.com/program-bot/mora-bot-data.git"} [**nms-example-rest**](http://npm.im/nms-example-rest): - original: git+https://bitbucket.org/smollweide/nms-example-rest.git - parsed: null [**notiontheory-basic-build**](http://npm.im/notiontheory-basic-build): - original: https://www.github.com/capnmidnight/notiontheory-basic-build - parsed: null [**npmts**](http://npm.im/npmts): - original: git+https://gitlab.com/pushrocks/npmts.git - parsed: null [**oc-trigger**](http://npm.im/oc-trigger): - original: https://git.oschina.net/steambap/trigger.git - parsed: null [**pgb-cordova-lib**](http://npm.im/pgb-cordova-lib): - original: git://git-wip-us.apache.org/repos/asf/cordova-lib.git - parsed: null [**pgb-plugman**](http://npm.im/pgb-plugman): - original: git://git-wip-us.apache.org/repos/asf/cordova-plugman.git - parsed: null [**pptxgenjs**](http://npm.im/pptxgenjs): - original: git+https://github.com/%3Agitbrent/PptxGenJS.git - parsed: null
zeke commented 8 years ago

Thanks for reporting. non-GitHub repository data is now kept intact as of 1.2.2