dogsheep / github-to-sqlite

Save data from GitHub to a SQLite database
https://github-to-sqlite.dogsheep.net/
Apache License 2.0
405 stars 43 forks source link

Set up a live demo Datasette instance #13

Closed simonw closed 4 years ago

simonw commented 4 years ago

I deployed https://github-to-sqlite-releases-j7hipcg4aq-uc.a.run.app/ by running this:

#!/bin/bash
# Fetch repos for simonw and dogsheep
github-to-sqlite repos github.db simonw dogsheep -a auth.json

# Fetch releases for the repos tagged 'datasette-io'
sqlite-utils github.db "
select full_name from repos where rowid in (
    select repos.rowid from repos, json_each(repos.topics) j
    where j.value = 'datasette-io'
)" --csv --no-headers | while read repo;
    do github-to-sqlite releases \
            github.db $(echo $repo | tr -d '\r') \
            -a auth.json;
        sleep 2;
    done;

And then deploying using this:

$ datasette publish cloudrun github.db \
  --title "github-to-sqlite releases demo" \
  --about_url="https://github.com/simonw/github-to-sqlite" \
  --about='github-to-sqlite' \
  --install=datasette-render-markdown \
  --install=datasette-json-html \
  --service=github-to-sqlite-releases

This should happen automatically for every release. I can run it once a day in Circle CI to keep the demo database up-to-date.

simonw commented 4 years ago

I'm going to do this with a scheduled GitHub Action in this repo.

simonw commented 4 years ago

I'll run the commits and issues and issue-comments commands in addition to the releases command.

simonw commented 4 years ago

Following these instructions: https://simonwillison.net/2020/Jan/21/github-actions-cloud-run/

simonw commented 4 years ago

Urgh this is such a mess! I should have done this on a branch / pull request to avoid polluting my main master history, but never mind.

simonw commented 4 years ago

Woohoo! https://github-to-sqlite-j7hipcg4aq-uc.a.run.app/

simonw commented 4 years ago

Most recently updated issues across all Dogsheep repos, with faceting: https://github-to-sqlite-j7hipcg4aq-uc.a.run.app/github/issues?_facet=repo&_facet=user&_facet=state&_facet=author_association&_facet=type&_sort_desc=updated_at

simonw commented 4 years ago

I pointed https://github-to-sqlite.dogsheep.net/ at it. May take a few minutes for the new certificate to provision though.

simonw commented 4 years ago

Next step: use a metadata.json file to add some extras. And add the datasette-render-markdown plugin as soon as I ship https://github.com/simonw/datasette-render-markdown/issues/2 (GFM support).

simonw commented 4 years ago

This is good enough for the 1.0 release.