Issues when creating runners in vcluster

Controller Version

0.25.1

Helm Chart Version

0.20.1

CertManager Version

1.1.0

Deployment Method

Helm

cert-manager installation

Yes cert-manager installed through proper procedure

Checks

[X] This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
[X] I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
[X] My actions-runner-controller version (v0.x.y) does support the feature
[X] I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue

Resource Definitions

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: krogertechnology-runner-deployment
spec:
  template:
    spec:
      image: docker-prod.registry.kroger.com/tools/actions-runner-dind:v2.292.0-ubuntu-20.04-no-proxy
      imagePullPolicy: IfNotPresent
      organization: krogertechnology
      labels:
      - vcluster-dev
      - aks-dev
      group: AKS Non-prod
      dockerdWithinRunnerContainer: true
      dockerRegistryMirror: https://docker.registry.kroger.com/
      resources:
        limits:
          cpu: "8"
          memory: "8Gi"
        requests:
          cpu: "1"
          memory: "8Gi"
      env:
      - name: DISABLE_RUNNER_UPDATE
        value: "true"

---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: krogertechnology-runner-deployment-autoscaler
spec:
  scaleTargetRef:
    name: krogertechnology-runner-deployment
  minReplicas: 1
  maxReplicas: 1
  metrics:
  - type: PercentageRunnersBusy
    scaleUpThreshold: '0.5'    # The percentage of busy runners at which the number of desired runners are re-evaluated to scale up
    scaleDownThreshold: '0.2'   # The percentage of busy runners at which the number of desired runners are re-evaluated to scale down
    scaleUpAdjustment: 10      # The scale up runner count added to desired count
    scaleDownAdjustment: 1    # The scale down runner count subtracted from the desired count

To Reproduce

1. Install the ARC using Helm
2. Create the runner deployments using the above manifest

Observe error

Describe the bug

We get this error Error from server (InternalError): error when creating "aks/runner-deployment-aks.yaml": Internal error occurred: failed calling webhook "mutate.runnerdeployment.actions.summerwind.dev": failed to call webhook: Post "https://actions-runner-controller-webhook.actions-runner-system.svc:443/mutate-actions-summerwind-dev-v1alpha1-runnerdeployment?timeout=10s": proxy error from 127.0.0.1:6443 while dialing 192.168.0.215:9443, code 500: 500 Internal Server Error

Describe the expected behavior

We expect the runner pods to be created.

Controller Logs

https://gist.github.com/shankarRaman/37954ad16b7725e2769b819576f7d0d7#file-shankar-arc-log

Runner Pod Logs

N/A

Additional Context

This is a vcluster setup using k3s

actions / actions-runner-controller