gravitational / teleport

The easiest, and most secure way to access and protect all of your infrastructure.
https://goteleport.com
GNU Affero General Public License v3.0
17.23k stars 1.73k forks source link

Original Error: *trace.AccessDeniedError id token claims did not match any allow rules: Github Machine ID #17939

Closed benarent closed 1 year ago

benarent commented 1 year ago

Expected behavior: I'm trying the new Github Machine ID code, but I'm running into this issue. It seems that my permissions are off, but it's unclear which it throwing this message.

Run ./tbot start -d --join-method=github --token=github-bot-2 --auth-server=teleport-11-ent.asteroid.earth:443 --oneshot --destination-dir=./certs --data-dir=./data
  ./tbot start -d --join-method=github --token=github-bot-[2](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:2) --auth-server=teleport-11-ent.asteroid.earth:44[3](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:3) --oneshot --destination-dir=./certs --data-dir=./data
  shell: /usr/bin/bash -e {0}
WARN [TBOT]      CLI parameters are overriding onboarding config from  config/config.go:377
INFO [TBOT]      Created directory "./data" config/destination_directory.go:132
INFO [TBOT]      Created directory "./certs" config/destination_directory.go:132
DEBU [TBOT]      Unable to load from current key "key-cert.pub", trying to migrate from old key "sshcert" identity/identity.go:[4](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:5)44
DEBU [TBOT]      Identity directory ./data is not found or empty and could not be loaded, will start from scratch: artifact "key-cert.pub" is unexpectedly empty in destination directory ./data tbot/tbot.go:320
INFO [TBOT]      Attempting to generate new identity from token tbot/renew.go:4[5](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:6)5
DEBU [AUTH]      Registering node to the cluster. auth-servers:[{teleport-11-ent.asteroid.earth:443 tcp }] auth/register.go:22[6](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:7)
DEBU [AUTH]      The first specified auth server appears to be a proxy. auth/register.go:232
INFO [AUTH]      Attempting registration via proxy server. auth/register.go:239
DEBU [CLIENT]    HTTPS client init(proxyAddr=teleport-11-ent.asteroid.earth:443, insecure=false) client/weblogin.go:259
DEBU [CLIENT]    Attempting https://teleport-11-ent.asteroid.earth:443/v1/webapi/host/credentials client/https_client.go:100
DEBU [AUTH]      Registration via proxy server failed. error:[
ERROR REPORT:
Original Error: *trace.AccessDeniedError id token claims did not match any allow rules
Stack Trace:

Caught:
    github.com/gravitational/teleport/lib/httplib/httplib.go:162 github.com/gravitational/teleport/lib/httplib.ConvertResponse
    github.com/gravitational/teleport/lib/client/https_client.go:105 github.com/gravitational/teleport/lib/client.(*WebClient).PostJSONWithFallback
    github.com/gravitational/teleport/lib/client/weblogin.go:544 github.com/gravitational/teleport/lib/client.HostCredentials
    github.com/gravitational/teleport/lib/auth/register.go:289 github.com/gravitational/teleport/lib/auth.registerThroughProxy
    github.com/gravitational/teleport/lib/auth/register.go:240 github.com/gravitational/teleport/lib/auth.Register
    github.com/gravitational/teleport/lib/tbot/renew.go:4[7](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:8)7 github.com/gravitational/teleport/lib/tbot.(*Bot).getIdentityFromToken
    github.com/gravitational/teleport/lib/tbot/tbot.go:331 github.com/gravitational/teleport/lib/tbot.(*Bot).initialize
    github.com/gravitational/teleport/lib/tbot/tbot.go:226 github.com/gravitational/teleport/lib/tbot.(*Bot).Run
    github.com/gravitational/teleport/tool/tbot/main.go:233 main.onStart
    github.com/gravitational/teleport/tool/tbot/main.go:152 main.Run
    github.com/gravitational/teleport/tool/tbot/main.go:47 main.main
    runtime/proc.go:250 runtime.main
    runtime/asm_amd64.s:1594 runtime.goexit
User Message: id token claims did not match any allow rules
] auth/register.go:243
INFO [AUTH]      Attempting registration with auth server. auth/register.go:239
WARN [AUTH]      Joining cluster without validating the identity of the Auth Server. This may open you up to a Man-In-The-Middle (MITM) attack if an attacker can gain privileged network access. To remedy this, use the CA pin value provided when join token was generated to validate the identity of the Auth Server. auth/register.go:405
DEBU [AUTH]      Registration with auth server failed. error:[
ERROR REPORT:
Original Error: *trace.ConnectionProblemError Post "https://teleport.cluster.local/v2/tokens/register": remote error: tls: internal error
Stack Trace:
    github.com/gravitational/teleport/lib/httplib/httplib.go:153 github.com/gravitational/teleport/lib/httplib.ConvertResponse
    github.com/gravitational/teleport/lib/auth/clt.go:254 github.com/gravitational/teleport/lib/auth.(*Client).PostJSON
    github.com/gravitational/teleport/lib/auth/clt.go:4[8](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:9)5 github.com/gravitational/teleport/lib/auth.(*Client).RegisterUsingToken
    github.com/gravitational/teleport/lib/auth/register.go:346 github.com/gravitational/teleport/lib/auth.registerThroughAuth
    github.com/gravitational/teleport/lib/auth/register.go:240 github.com/gravitational/teleport/lib/auth.Register
    github.com/gravitational/teleport/lib/tbot/renew.go:477 github.com/gravitational/teleport/lib/tbot.(*Bot).getIdentityFromToken
    github.com/gravitational/teleport/lib/tbot/tbot.go:331 github.com/gravitational/teleport/lib/tbot.(*Bot).initialize
    github.com/gravitational/teleport/lib/tbot/tbot.go:226 github.com/gravitational/teleport/lib/tbot.(*Bot).Run
    github.com/gravitational/teleport/tool/tbot/main.go:233 main.onStart
    github.com/gravitational/teleport/tool/tbot/main.go:152 main.Run
    github.com/gravitational/teleport/tool/tbot/main.go:47 main.main
    runtime/proc.go:250 runtime.main
    runtime/asm_amd64.s:15[9](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:10)4 runtime.goexit
User Message: Post "https://teleport.cluster.local/v2/tokens/register": remote error: tls: internal error] auth/register.go:243

ERROR REPORT:
Original Error: trace.aggregate id token claims did not match any allow rules, Post "https://teleport.cluster.local/v2/tokens/register": remote error: tls: internal error
Stack Trace:
    github.com/gravitational/teleport/lib/auth/register.go:249 github.com/gravitational/teleport/lib/auth.Register
    github.com/gravitational/teleport/lib/tbot/renew.go:477 github.com/gravitational/teleport/lib/tbot.(*Bot).getIdentityFromToken
    github.com/gravitational/teleport/lib/tbot/tbot.go:331 github.com/gravitational/teleport/lib/tbot.(*Bot).initialize
    github.com/gravitational/teleport/lib/tbot/tbot.go:226 github.com/gravitational/teleport/lib/tbot.(*Bot).Run
    github.com/gravitational/teleport/tool/tbot/main.go:233 main.onStart
    github.com/gravitational/teleport/tool/tbot/main.go:[15](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:16)2 main.Run
    github.com/gravitational/teleport/tool/tbot/main.go:47 main.main
    runtime/proc.go:[25](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:26)0 runtime.main
    runtime/asm_amd64.s:1594 runtime.goexit
User Message: id token claims did not match any allow rules, Post &#[34](https://github.com/asteroid-earth/example-gh-actions-machineid/actions/runs/3353173185/jobs/5555796861#step:6:35);https://teleport.cluster.local/v2/tokens/register": remote error: tls: internal error
Error: Process completed with exit code 1.

Current behavior:

Bug details:

zmb3 commented 1 year ago

Please share the join token YAML (with any secrets redacted).

strideynet commented 1 year ago

I think we can probably do something to improve the logging on the server-side, but I'm not sure I want to leak information about which claim doesn't match to a client. In this case, if you share the token YAML we can probably work out what's not correct here :)

benarent commented 1 year ago

My token was created like this

kind: token
version: v2
metadata:
  name: github-bot-2
  expires: "3000-01-01T00:00:00Z"
spec:
  roles: [Bot, Node, Db, App, Kube]
  join_method: github
  bot_name: robot-2
  github:
    allow:
      - repository: asteroid-earth/example-gh-actions-machineid
        repository_owner: benarent

Then added via tctl bots with tctl bots add robot-2 --token=github-bot-2 --roles=access --logins=root

I created my first token with Terraform, that seemed to register but tried the YAML to remove any other steps.

resource "teleport_provision_token" "github_bot" {
  spec = {
    roles       = ["Bot"]
    join_method = "github"
    # tctl bots add github-bot --token=github-bot --roles=access --logins=root
    bot_name    = "github-bot"
    github: {
      "allow": [
        {
          "repository": "asteroid-earth/example-gh-actions-machineid",
          "repository_owner": "benarent"
        }
      ]
    }
  }

  metadata = {
    name        = "github-bot"
    expires     = "3000-12-12T12:05:23Z"
    description = "testing the new provider sensitive values"
  }
}

Github Actions Code

name: Demo
on:
  push:
    branches:
      - main

jobs:
  demo:
    permissions:
      # This is required or tbot will not be able to authenticate
      id-token: write
      contents: read
    name: demo
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Fetch tsh and tbot
        run: wget https://cdn.teleport.dev/teleport-v11.0.1-linux-amd64-bin.tar.gz
      - name: Install tsh
        run: tar -xvf teleport-v11.0.1-linux-amd64-bin.tar.gz
      - name: Install tsh 
        run: sudo cp ./teleport/tbot ./tbot
      - name: Fetch credentials a
        run: ./tbot start -d --join-method=github --token=github-bot-2 --auth-server=teleport-11-ent.asteroid.earth:443 --oneshot --destination-dir=./certs --data-dir=./data
      - run: ls ./certs
      - name: List nodes
        run: ./tsh -i ./certs/identity --proxy teleport-11-ent.asteroid.earth:443 ls
strideynet commented 1 year ago

repository_owner in this case would be asteroid-earth, not benarent.

It's a little redundant really as in most cases, folks are going to be providing repository which includes this already.

strideynet commented 1 year ago

Raised https://github.com/gravitational/teleport/issues/17948 for improving these errors.

I'll close this down - feel free to message me on Slack if you still can't get this working.

sec0ndhand commented 1 year ago

I'm getting,

id token claims did not match any allow rules

using the this walkthrough.

I posted this in slack, but this might put relevant information here if it needs to be fixed in code.

I tried this in one repo [my-org]/first-api and it didn't work. I then recreated keys, and bots multiple times but still got the error above when running in the first-api repo.

Then created a brand new repo [my-org]/teleport-test and used your sample action, and it worked! I was thrilled. So I copied the same action and ran it with no modifications in [my-org]/first-api and it still failed with the above message. I even created a second token and bot, and re-ran it, receiving the same error from the first-api repo.

Is there a reason this would work differently between 2 repos?

My token creation file:

kind: token
version: v2
metadata:
  name: github-token
  expires: "2100-01-01T00:00:00Z"
spec:
  roles: [Bot]
  join_method: github
  bot_name: github-action
  github:
    allow:
      - repository: [my-org]/first-api
      - repository: [my-org]/second-api
      - repository: [my-org]/teleport-test

My GitHub action running in both teleport-test and first-api:

# This is a basic workflow to help you get started.
# It will take the following action whenever a push is made to the "master" branch.
name: Testing Teleport
run-name: We're just seeing if this even works
on:
  push:
    branches:
      - master
      - main
      - development
  workflow_dispatch:
    branches:
      - master
      - main
      - development
jobs:
  demo:
    permissions:
      # The "id-token: write" permission is required or Machine ID will not be able to authenticate with the cluster.
      id-token: write
      contents: read
    # The name of the workflow, and the Linux distro to be used to perform the required steps.
    name: guide-demo
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Fetch Teleport binaries
        uses: teleport-actions/setup@v1
        with:
          version: 12.1.1
      - name: Fetch credentials using Machine ID
        id: auth
        uses: teleport-actions/auth@v1
        with:
          # Use the address of the auth/proxy server for your own cluster.
          proxy: teleport.mydomain.com:443
          # Use the name of the join token resource you created in step 1.
          token: github-token
          # Specify the length of time that the generated credentials should be
          # valid for. This is optional and defaults to "1h"
          certificate-ttl: 1h
          # Enable the submission of anonymous usage telemetry.
          anonymous-telemetry: 1
      - name: Write file to remote
        # Enters a command from the cluster, in this case "tsh ls" using Machine ID credentials to list remote SSH nodes.
        run: tsh -i ${{ steps.auth.outputs.identity-file }} ssh user@hostname "echo $GITHUB_SHA >> ~/github_run_log"