Failed to get new registration token while installing ARC org wide for enterprise GITHUB

itsmekarthikreddy commented 2 months ago

Checks

[X] I've already read https://github.com/actions/actions-runner-controller/blob/master/TROUBLESHOOTING.md and I'm sure my issue is not covered in the troubleshooting guide.
[X] I'm not using a custom entrypoint in my runner image

Controller Version

0.27.6

Helm Chart Version

0.23.7

CertManager Version

1.8.0

Deployment Method

Helm

cert-manager installation

Yes Installed as per ARC Doc

Checks

[X] This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
[X] I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
[X] My actions-runner-controller version (v0.x.y) does support the feature
[X] I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
[X] I've migrated to the workflow job webhook event (if you using webhook driven scaling)

Resource Definitions

---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: k8s-runners-autoscaler
  namespace: XXXX-actions-runners
spec:
  scaleTargetRef:
    name: k8s-runners
  scaleDownDelaySecondsAfterScaleOut: 300
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
    repositoryNames:
    - XXX/XXX
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: Runner
metadata:
  name: k8s-single-runner
  namespace: XXXX-actions-runners
spec:
  repository: XXXX/XXXX
  env: []
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: k8s-runners
  namespace: XXXX-actions-runners
spec:
  template:
    spec:
      repository: XXXX/XXXX
      labels:
        - XXX-core-stack
      env: []

To Reproduce

Create GIT HUB app as per https://github.com/actions/actions-runner-controller/blob/master/docs/authenticating-to-the-github-api.md
Create secret controller-manager in actions-runners-system with above details and pem key

Describe the bug

Failed to get new registration token

Describe the expected behavior

Runner should be created successfully.

Whole Controller Logs

https://gist.github.com/itsmekarthikreddy/18c5112333acaee2c54f0945b0f87222

Whole Runner Pod Logs

https://gist.github.com/itsmekarthikreddy/18c5112333acaee2c54f0945b0f87222

Additional Context

No response

github-actions[bot] commented 2 months ago

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.

itsmekarthikreddy commented 2 months ago

FYI , I have created a standard cluster not auto pilot in gcp

actions-actions-runner-controller manger pod logs :


github.com/actions-runner-controller/actions-runner-controller/controllers.(*RunnerReconciler).Reconcile
    /workspace/controllers/runner_controller.go:189
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:214
2024-05-02T08:44:52.580Z    ERROR   controller-runtime.manager.controller.runner-controller Reconciler error    {"reconciler group": "actions.summerwind.dev", "reconciler kind": "Runner", "name": "k8s-runners-4xf79-lhnrk", "namespace": "actions", "error": "failed to create registration token: POST https://api.github.com/repos/XXXX-XXX/XXXX/actions/runners/registration-token: 401 Bad credentials []"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:214
2024-05-02T08:44:52.580Z    DEBUG   controller-runtime.manager.events   Warning {"object": {"kind":"Runner","namespace":"actions","name":"k8s-runners-4xf79-lhnrk","uid":"5a59b3b4-8186-4450-b05d-94c0108400e3","apiVersion":"actions.summerwind.dev/v1alpha1","resourceVersion":"8139"}, "reason": "FailedUpdateRegistrationToken", "message": "Updating registration token failed"}
2024-05-02T08:44:52.774Z    ERROR   actions-runner-controller.runner    Failed to get new registration token    {"runner": "k8s-single-runner", "error": "failed to create registration token: POST https://api.github.com/repos/XXXX-XXX/XXXX/actions/runners/registration-token: 401 Bad credentials []"}
github.com/actions-runner-controller/actions-runner-controller/controllers.(*RunnerReconciler).Reconcile```

cdwbrad commented 1 month ago

@itsmekarthikreddy I'm currently experiencing a similar issue. Were you able to resolve? And if so, how?

adamyodinsky commented 3 weeks ago

For me it was the issue below

I used "spec.template.spec.repository" field instead of "spec.template.spec.organization" in the runnerDeployment, when using github app creds for an organization. when changes it to organization everything worked.

The logs are not very helpful to understand what is wrong, and pin point the issue did took some time.

Hope it will help someone.

actions / actions-runner-controller