[API] Creating auth key via API (OAuth) returns "exactly one capability scope must be populated"

colophonemes commented 2 weeks ago

What is the issue?

I have a script that provisions new headless Tailscale services in our network. It first uses an OAuth client to get an API access token, and then uses this to register a new device (i.e. create an auth key using the https://api.tailscale.com/api/v2/tailnet/-/keys endpoint).

The script has worked without issue at least up until a few weeks ago (we don't run it super frequently but I have a successful run logged on 19 April). However, now when I attempt to register the new device the API returns the following:

{"message":"exactly one capability scope must be populated"}

I have:

checked that my OAuth key has devices:write scope
tried generating new OAuth keys (with devices:write scope)
diffed the payload I'm POSTing to the /keys endpoint against the sample provided here
sending empty payloads, payloads with typos, payloads without expirySeconds and description, different values for expirySeconds, an empty JSON object for devices etc
sending an invalid value for the API access token (which fails with {"message":"API token invalid"} as expected)
checked that curl is indeed sending the JSON POST data correctly with --trace-ascii -
checked that the API access token that I get can actually make valid requests (e.g. I can successfully issue GET requests to /keys)

Steps to reproduce

Here is the relevant part of the script I'm using:

# https://github.com/tailscale/tailscale/blob/main/api.md#create-auth-key
# Use jq to parse the heredoc and then we get JSON validation for free
auth_key_payload=$(jq <<EOF
{
  "capabilities": {
    "devices": {
      "create": {
        "reusable": false,
        "ephemeral": true,
        "preauthorized": true,
        "tags": $tags
      }
    }
  },
  "expirySeconds": 0,
  "description": "$TAILSCALE_HOSTNAME container access"
}
EOF
)

# Get an auth token for use with the API using an OAuth client
# See https://tailscale.com/kb/1215/oauth-clients
echo "Getting Tailscale API access token..."
ts_api_token=$(curl --silent -d "client_id=$TS_API_CLIENT_ID" -d "client_secret=$TS_API_CLIENT_SECRET" \
            "https://api.tailscale.com/api/v2/oauth/token" | jq -r '.access_token')

if [ "$ts_api_token" = "null" ]; then
  echo "Tailscale API returned invalid API access token!"
  exit 1
fi

# Generate an auth key using our API key and payload
echo "Getting Tailscale auth key..."
echo "[auth key payload]:"
echo "$auth_key_payload"
echo ""
ts_auth_key_res=$(curl --silent -X POST --user "$ts_api_token:" \
    https://api.tailscale.com/api/v2/tailnet/-/keys \
    --data-binary "$auth_key_payload")

echo "Auth key result:"
echo "$ts_auth_key_res"
echo ""

ts_auth_key=$(
  echo "$ts_auth_key_res" |
  jq -r '.key'
)

if [ "$ts_auth_key" = "null" ]; then
  echo "Tailscale API returned invalid auth key!"
  exit 1
fi

And the output:

❯ docker run --name ts-test --env-file docker/.env tailscale-oauth
Getting Tailscale API access token...
Getting Tailscale auth key...
[auth key payload]:
{
  "capabilities": {
    "devices": {
      "create": {
        "reusable": false,
        "ephemeral": true,
        "preauthorized": true,
        "tags": [
          "tag:sandbox"
        ]
      }
    }
  },
  "expirySeconds": 0,
  "description": "my-test-service container access"
}

Auth key result:
{"message":"exactly one capability scope must be populated"}

Tailscale API returned invalid auth key!

Are there any recent changes that introduced the issue?

Unsure, but script previously worked, and now does not, so I suspect some change in the API validation or parsing logic.

OS

Linux

OS version

Docker image: tailscale/tailscale:v1.44

Tailscale version

v1.44, v1.58, v1.60.1

Other software

No response

Bug report

BUG-b30a867132611709fba6460205d9f1c8ea88caa94d13f76a0d0b1ff7cf8a86c4-20240429142037Z-0ce6e4301266f44f

kelivel commented 2 weeks ago

Hi @colophonemes, thanks for flagging this. We're actively investigating and should have a fix shipped soon!

kelivel commented 2 weeks ago

Thanks again for reporting this. The fix for this should now be live. Please let us know if you continue to run into any issues.

colophonemes commented 2 weeks ago

Amazing, seems to be working now.

I notice that you've updated the docs to ask for Content-type: application/json, but my script which is based on the old examples (i.e. sends them as application/x-www-form-urlencoded) now works. Is the recommendation to switch to application/json?

willnorris commented 2 weeks ago

Yes, the recommendation is to switch to using application/json explicitly. Previously, we were pretty lax about the content-type, but we recently made a change that was a bit more strict about how the content-type is interpreted, which is what caused the failures. We've switched back to be more lax, particularly because curl -d defaults to using application/x-www-form-urlencoded if no other content type is specified. And since that's such a common way of calling the API, it makes sense to accept that, even if it is technically wrong.

Any users that were previously setting the application/json content type, either manually with curl or whatever client library they were using, shouldn't have run into this issue.

willnorris commented 2 weeks ago

And I say that setting the content-type is the recommendation only because it's just good practice in general. We've added tests to our API to make sure that we continue to be lax about our own treatment of the content-type, so for the Tailscale API in particular, it shouldn't really matter. But it's still good practice anyway.

colophonemes commented 2 weeks ago

Great, that's super helpful. I had assumed (based on the previous version of the docs not mentioning it) that the intention was actually to send application/x-www-form-urlencoded (although thinking about it, this doesn't actually really make sense), but I'll update my scripts to add this now.

Thanks again for the speedy response here!

tailscale / tailscale