Plugin data collection proposal

[jonathantneal]

I’d like to recommend an updated list of data we should collect for plugins and how we should collect it.

1. Collect the target NPM name:
   • process.argv[2], the first argument passed to the script.
2. Collect entry data from NPM:
   • https://registry.npmjs.org/${ name }/latest:
     • Collect author, the primary author.
     • Collect dependencies, all/plugin dependencies.
     • Collect keywords, additional search terms.
     • Collect categories, plugin category terms:

       keywords.filter(
           (keyword) => /^postcss-plugin-(.+)/.test(keyword)
       ).map(
           (keyword) => keyword.replace(
               /^postcss-plugin-(.+)/,
               '$1'
           )
       ).reduce(
           (categories, keyword) => categories.concat(keyword.split('-')),
           []
       )

     • Collect repo, the GitHub repository path:

       repository.url.replace(
           /^git\+https:\/\/github\.com\/(.+)\.git$/,
           '$1'
       )

   • https://api.npmjs.org/downloads/range/1000-01-01:2100-01-01/${ name }:
     • Collect downloads, the total number of downloads:

       downloads.reduce(
           (count, download) => count + download.downloads,
           0
       )

3. Collect entry data from GitHub:
   • https://api.github.com/repos/${ repo }:
     • Collect stars, the number of stars (from stargazers_count).
     • Collect forks, the number of forks.
     • Collect issues, the number of open issues (from open_issues_count).
   • https://api.github.com/repos/${ repo }/contributors:
     • Collect contributors, the list of contributors:
       • Collect each user, a contributor’s username (from login).
       • Collect each contributions, a contributor’s number of contributions.
       • Collect each avatar, a contributor’s avatar (from avatar_url).
4. Read the database from the JSON file.
5. Push the plugin entry data to the database by name.
6. Sort the database by names.
7. Write the database back to the JSON file.

[jonathantneal]

@ben-eb had a suggestion to periodically query the npm registry ourselves. That would add a minor adjustment to the instructions.

1. Collect the list of PostCSS plugins from NPM:
   • https://registry.npmjs.org/-/_view/byKeyword?startkey=[%22postcss-plugin%22]&endkey=[%22postcss-plugin%22,{}]&group_level=2:

     result.rows.map(
         (row) => row.key[1]
     )

2. Collect each entry data from NPM...

We might also use the dependencies object to validate whether PostCSS is summoned.