search

pdehaan / repo-health-report

Issue tracker stats for repos.
https://repo-health-report.now.sh/
Mozilla Public License 2.0
3 stars 0 forks source link

Figure out data structure #4

Closed pdehaan closed 4 years ago

pdehaan commented 4 years ago

This might be related to the solution of static/Eleventy vs dynamic/Express, but currently we're storing scraped data in src/_data/{owner}/{repo}/{date}.json in 11ty which makes these global data files.

Alternatively, we could do some page specific data files, but that can be a bit messy. For example, we might have the following two files for each repo/update:

src/github/mozilla/blurts-server/2020-03-30.njk src/github/mozilla/blurts-server/2020-03-30.11tydata.json

But then that makes comparing time series pretty complicated.

Another solution is to figure out the absolute bare minimum number of data we care about for a repo and then store JUST that data in a big data file. So maybe src/_data/{owner}/{repo}.json which has all data, where we just append daily results to the end of the file. Makes it much easier to look at trends over time since everything is in a single file.

pdehaan commented 4 years ago

More random thoughts... I don't like dashes in data filenames. Stuff like "blurts-server" gets tricky to consume. Not sure if we simplify and do:

src/_data/{{repo}}.json

Where the repo name is slugified and converts hyphens into underscores (ie: "blurts_server.json"). Also, need to figure out if we can ideally do this with a single data file per project. So blurts_server.json just has an object/array of the stats that we need, and we just append to that. Makes looping and consuming the data much easier.

pdehaan commented 4 years ago

I think we have this in a state that we can call "Good Enough"(tm). Currently repos are just a one page thing and I haven't needed to do a complex historical archive in src/_data yet.

/index.njk
/github/
  /mozilla/
    /{repo1}/
      /{repo1}.11tydata.json
      /index.njk

  /mozilla-services/
    /{repo2}/
      /{repo2}.11tydata.json
      /index.njk

  /mozilla-mobile/
    /{repo3}/
      /{repo3}.11tydata.json
      /index.njk

/bugzilla/
  /tbd/

We can put 11ty data files in any level of folder that we think we need. Layouts can be shared globally or set at a /github/ vs /bugzilla/ level, and we can do any customizations we need in the relevant project's index.njk file.

Closing this until it fails catastrophically and we need to revisit.