Add support for mono repositories. - Githubissues

npms-io / npms-analyzer

The analyzer behind https://npms.io

MIT License

319 stars 38 forks source link

Add support for mono repositories. #147

Closed satazor closed 7 years ago

satazor commented 7 years ago

The downloaded object that is returned by downloaders has the following shape:

{
   type: 'github|git|npm',  // Downloader type
   dir: '/tmp/random',  // Directory where the downloader saved stuff
   packageJson: {}, // Downloaded package json
   gitRef: '',  // Git ref if downloader was github or git
}

Though, in mono repositories, the package is not downloaded.dir but a directory inside it. Therefore, packageDir was added that basically points to the correct package directory. Note that for standard repositories, downloaded.dir === downloaded.packageDir.

The collectors were updated to use downladed.packageDir whenever appropriate, thanks @bebraw!

Extraneous stuff in this PR:

Remove unnecessary refOverrides since it's no longer necessary: this was created to circumvent issues solved by this PR
Remove unused repositorySize
Invalid package name always throws.
Fix various lang typos.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.063% when pulling 948c89199751579b3efdcfa2215566291316dc91 on monorepos into 69cade6ee8061270c923dfe3b7548ebb72023ff3 on master.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.063% when pulling f12988cb687db45329e3cc3333fa41a4aef214bc on monorepos into 69cade6ee8061270c923dfe3b7548ebb72023ff3 on master.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.063% when pulling 46ce081b1419fb4ef3ecd7d82248c3a43ef50cca on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.063% when pulling 50319a7e944fa69b8f547ef8fe3aa5eaef697ecc on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.063% when pulling e7a10506e82d035b62db83f71445ca135c523eef on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.

satazor commented 7 years ago

Some tests comparing before vs after on some mono repositories.

react-router: 0.39 vs 0.90!
babel-jest: 0.34 vs 0.84!
create-react-app: 0.27 vs 0.76!
makestatic: 0.23 vs 0.63
react-scripts: 0.26 vs 0.76!
babel: 0.38 vs 0.73!
pouchdb: 0.30 vs 0.78!

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.059% when pulling e74dd7039b560ab36a37cf3c78a334ed3508893b on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.

satazor commented 7 years ago

Do not merge it yet, TODO:

~~Normalize & fix repository field that have this pattern: https://github.com/babel/babel/issues/5574~~
~~Consider using deep compact or deep assign in mergePackageJson because of empty props, such as scripts in bower package~~

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.095% when pulling 178ecfe053795f7f359fd23d3a2257acd92db9c8 on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.095% when pulling 178ecfe053795f7f359fd23d3a2257acd92db9c8 on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.103% when pulling d8b392bdca6a17bb9843a469d92f1a00274c131a on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.103% when pulling a0bea6b129496ff040aac4874731071ed9181683 on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.

coveralls commented 7 years ago

Coverage increased (+0.3%) to 97.103% when pulling 5fee4a17daa655ee738ae62f300e22181c15e372 on monorepos into e81892bb6fa67dce5ab448bd4570cbeb3f4fcf39 on master.