One-off Authorization header auth plugin

[simonw]

Need a one-off plugin for token authentication so that #5 can download the existing database.

Originally posted by @simonw in https://github.com/simonw/big-local-datasette/issues/5#issuecomment-611867772

[simonw]

Shipped a fix for this related issue in datasette-github-auth~=0.12: https://github.com/simonw/datasette-auth-github/issues/59

[simonw]

Token secret for prod will be 1af41451fd8c9ef1927e34889c5d74ac for the moment.

[simonw]

datasette publish cloudrun *.db \
    --service=biglocal \
    --install=datasette-auth-github \
    --plugin-secret datasette-auth-github client_id f4084e44a25d23ce1c99 \
    --plugin-secret datasette-auth-github client_secret \
8a428d8092e6eac84d1eb6ada315a4e016b9e34f \
    --plugin-secret datasette-auth-github allow_org biglocalnews \
    --memory=2Gi \
    --plugins-dir=plugins \
    --plugin-secret token-auth secret 1af41451fd8c9ef1927e34889c5d74ac

[simonw]

This isn't working - it redirects to GitHub - and I'm not sure why:

curl -v -H 'Authorization: Bearer 1af41451fd8c9ef1927e34889c5d74ac' https://biglocal.datasettes.com/-/versions.json

[simonw]

It works on my local machine though.

I thought it might be down to the order in which the plugins are loaded - but the custom plugin has @hookimpl(trylast=True) while datasette-github-auth doesn't so the order should be predictable in both cases.

[simonw]

The line I use locally to run a server that works is:

TOKEN_AUTH_SECRET=hello \
DATASETTE_AUTH_GITHUB_CLIENT_ID=foo \
DATASETTE_AUTH_GITHUB_CLIENT_SECRET=bar \
DATASETTE_AUTH_GITHUB_ALLOW_ORG=oo \
datasette -m metadata.json *.db -p 8007 --plugins-dir=plugins

Then:

~ $ curl 'http://127.0.0.1:8007/-/plugins.json' -H 'Authorization: Bearer hello' | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   171    0   171    0     0  60962      0 --:--:-- --:--:-- --:--:-- 85500
[
  {
    "name": "datasette-auth-github",
    "static": false,
    "templates": true,
    "version": "0.12"
  },
  {
    "name": "token_auth.py",
    "static": false,
    "templates": false,
    "version": null
  }
]

[simonw]

Figured it out: I forget the -m metadata.json when I ran the deploy, which means that auth on this line was None:

https://github.com/simonw/big-local-datasette/blob/730d12f4bae66381848313817ff91e39fc372630/plugins/token_auth.py#L26-L30

[simonw]

That didn't fix it, but did highlight a new bug.

https://biglocal.datasettes.com/-/metadata reports this:

{
    "title": "Big Local News open projects",
    "source_url": "https://biglocalnews.org/",
    "source": "Big Local News",
    "plugins": {
        "datasette-auth-github": {
            "client_id": {
                "$env": "DATASETTE_AUTH_GITHUB_CLIENT_ID"
            },
            "client_secret": {
                "$env": "DATASETTE_AUTH_GITHUB_CLIENT_SECRET"
            },
            "allow_org": {
                "$env": "DATASETTE_AUTH_GITHUB_ALLOW_ORG"
            }
        },
        "token-auth": {
            "secret": {
                "$env": "TOKEN_AUTH_SECRET"
            }
        }
    }
}

BUT... the metadata.json I attempted to publish was this:

{
    "title": "Big Local News open projects",
    "source_url": "https://biglocalnews.org/",
    "source": "Big Local News",
    "plugins": {
        "token-auth": {
            "secret": {
                "$env": "TOKEN_AUTH_SECRET"
            },
            "auth": {
                "name": "token-bot"
            }
        },
        "datasette-auth-github": {
            "client_id": {
                "$env": "DATASETTE_AUTH_GITHUB_CLIENT_ID"
            },
            "client_secret": {
                "$env": "DATASETTE_AUTH_GITHUB_CLIENT_SECRET"
            },
            "allow_org": {
                "$env": "DATASETTE_AUTH_GITHUB_ALLOW_ORG"
            }
        }
    }
}

The "auth" key is missing from production. It looks like the plugin secrets mechanism wiped it out somehow.

For the moment I will by hard-coding the key in the metadata.

[simonw]

datasette publish cloudrun *.db --service=biglocal --install=datasette-auth-github --plugin-secret datasette-auth-github client_id f4084e44a25d23ce1c99 --plugin-secret datasette-auth-github client_secret 8a428d8092e6eac84d1eb6ada315a4e016b9e34f --plugin-secret datasette-auth-github allow_org biglocalnews --memory=2Gi --plugins-dir=plugins -m metadata.json

[simonw]

This is REALLY weird. https://biglocal.datasettes.com/-/metadata now reports:

{
    "title": "Big Local News open projects",
    "source_url": "https://biglocalnews.org/",
    "source": "Big Local News",
    "plugins": {
        "datasette-auth-github": {
            "client_id": {
                "$env": "DATASETTE_AUTH_GITHUB_CLIENT_ID"
            },
            "client_secret": {
                "$env": "DATASETTE_AUTH_GITHUB_CLIENT_SECRET"
            },
            "allow_org": {
                "$env": "DATASETTE_AUTH_GITHUB_ALLOW_ORG"
            }
        }
    }
}

The "token-auth" section is completely gone!

[simonw]

It's a Datasette core bug:

https://github.com/simonw/datasette/blob/af9cd4ca64652fae262e6f7b5d201f6e0adc989b/datasette/publish/cloudrun.py#L98-L109

        if plugin_secret:
            extra_metadata["plugins"] = {}
            for plugin_name, plugin_setting, setting_value in plugin_secret:
                environment_variable = (
                    "{}_{}".format(plugin_name, plugin_setting)
                    .upper()
                    .replace("-", "_")
                )
                environment_variables[environment_variable] = setting_value
                extra_metadata["plugins"].setdefault(plugin_name, {})[
                    plugin_setting
                ] = {"$env": environment_variable}

[simonw]

https://github.com/simonw/datasette/issues/724

[simonw]

I fixed that bug in Datasette maser, so deploying branch=d349d57cdf3d577afb62bdf784af342a4d5be660 should fix the problem here.

datasette publish cloudrun *.db \
  --service=biglocal \
  --install=datasette-auth-github \
  --plugin-secret datasette-auth-github client_id f4084e44a25d23ce1c99 \
  --plugin-secret datasette-auth-github client_secret 8a428d8092e6eac84d1eb6ada315a4e016b9e34f \
  --plugin-secret datasette-auth-github allow_org biglocalnews \
  --memory=2Gi \
  --plugins-dir=plugins \
  -m metadata.json \
  --branch=d349d57cdf3d577afb62bdf784af342a4d5be660

[simonw]

That didn't work... because the version that matters is the Datasette version used to run datasete publish cloudrun, NOT the version that is installed on the server.

pip install https://github.com/simonw/datasette/archive/d349d57cdf3d577afb62bdf784af342a4d5be660.zip and then the deploy command minus the --branch option should do it.

[simonw]

https://biglocal.datasettes.com/-/metadata now shows the correct merged settings:

{
    "title": "Big Local News open projects",
    "source_url": "https://biglocalnews.org/",
    "source": "Big Local News",
    "plugins": {
        "token-auth": {
            "secret": "1af41451fd8c9ef1927e34889c5d74ac",
            "auth": {
                "name": "token-bot"
            }
        },
        "datasette-auth-github": {
            "client_id": {
                "$env": "DATASETTE_AUTH_GITHUB_CLIENT_ID"
            },
            "client_secret": {
                "$env": "DATASETTE_AUTH_GITHUB_CLIENT_SECRET"
            },
            "allow_org": {
                "$env": "DATASETTE_AUTH_GITHUB_ALLOW_ORG"
            }
        }
    }
}

And this curl command works:

$ curl -s -H 'Authorization: Bearer 1af41451fd8c9ef1927e34889c5d74ac' https://biglocal.datasettes.com/-/versions.json | jq .
{
  "python": {
    "version": "3.8.2",
    "full": "3.8.2 (default, Mar 31 2020, 15:23:55) \n[GCC 8.3.0]"
  },
  "datasette": {
    "version": "0.39"
  },
  "asgi": "3.0",
  "uvicorn": "0.11.3",
  "sqlite": {
...