Only deploy to Cloud Run if data has changed

[simonw]

Follow on from #5.

    - name: Set variables to decide if we should deploy
      id: decide_variables
      run: |-
        echo "##[set-output name=latest;]$(datasette inspect covid.db | jq '.covid.hash' -r)"
        echo "##[set-output name=deployed;]$(curl -s https://covid-19.datasettes.com/-/databases.json | jq '.[0].hash' -r)"
    - name: Set up Cloud Run
      if: contains(github.event.head_commit.message, 'DEPLOY') || steps.decide_variables.outputs.latest != steps.decide_variables.outputs.deployed

[simonw]

Re-enable cron that I disabled in https://github.com/simonw/big-local-datasette/commit/10222fca75613454c7dfb3197e0d3872c33a89ea once I fix this.

[simonw]

I'll use this trick:

Maybe hash all of the .db files: md5 *.db | md5 - and compare to the version_note? Originally posted by @simonw in https://github.com/simonw/big-local-datasette/issues/5#issuecomment-611866437

[simonw]

I can get the current version note using:

curl -s -H "Authorization: Bearer $DATASETTE_TOKEN" https://biglocal.datasettes.com/-/versions.json | jq '.datasette.note' -r

This returns a blank string if the version note has not been set yet.

[simonw]

    - name: Set variables to decide if we should deploy
      id: decide_variables
      env:
        DATASETTE_TOKEN: ${{ secrets.DATASETTE_TOKEN }}
      run: |-
        echo "##[set-output name=latest_db_hash;]$(md5 *.db | md5)"
        echo "##[set-output name=deployed_db_hash;]$(curl -s -H "Authorization: Bearer $DATASETTE_TOKEN" https://biglocal.datasettes.com/-/versions.json | jq '.datasette.note' -r)"

[simonw]

It's working now! See https://github.com/simonw/big-local-datasette/runs/581797891?check_suite_focus=true

[simonw]

Tweaked md5 checksums in https://github.com/simonw/big-local-datasette/commit/7364e5c97e30d5e4c7316477fd121a10117ac258

I am getting 'acd010291929a9c1f0aac78f66872e5a -' at the moment.

This should remove the trailing hyphen.

https://stackoverflow.com/a/3679329/6083

[simonw]

https://biglocal.datasettes.com/-/versions now reports:

{
    "python": {
        "version": "3.8.2",
        "full": "3.8.2 (default, Mar 31 2020, 15:23:55) \n[GCC 8.3.0]"
    },
    "datasette": {
        "version": "0.39",
        "note": "579aa25c17aefcd2b0f1beb6412cd692"
    }

[simonw]

If c12b82f436bb92a45f11760711f58b488759e050 skips the deploy then I'll know this is working.

[simonw]

Actually that won't skip the deploy because we deploy on every push: https://github.com/simonw/big-local-datasette/blob/c12b82f436bb92a45f11760711f58b488759e050/.github/workflows/deploy.yml#L57 I'll have to wait for the cron to run to see if this works.

[simonw]

Looks like it's changing the md5 every time, I think because the biglocal.db database will have different content (different signed hashes) every time.

Fix is to hash everything except for that file. Not sure how to do that yet, so I'll disable the cron again for the moment.

[simonw]

https://unix.stackexchange.com/a/190581

This pattern should work:

!(biglocal*).db

[simonw]

Oops, that broke. Maybe set the md5 in a separate variable on the line before the output variable echo?

[simonw]

Still getting this error:

Run export LATEST_DB_HASH=$(md5sum -b !(biglocal*).db | md5sum | cut -f 1 -d ' ')
/home/runner/work/_temp/df357cea-1d46-43d9-88c5-d09da5bdc1d6.sh: command substitution: line 2: syntax error near unexpected token `('

[simonw]

I will close this as soon as the cron runs and skips a deploy.

[simonw]

https://github.com/simonw/big-local-datasette/runs/583631574?check_suite_focus=true seems to be generating the right hashes now:

[simonw]

This worked! https://github.com/simonw/big-local-datasette/runs/583754515?check_suite_focus=true