[concurrency] Reruns a pipelinerun hang in state PipelineRunPending

[jwitrick]

Expected Behavior

When using the rerun pipelinerun functionality, the newly created pipelinerun should automatically start running (except in the situation of concurrency rule).

Actual Behavior

When there are NO running pipelineruns, when a user reruns a non running pipelinerun (either in state Completed or Failed) the newly created pipelinerun starts in status PipelineRunPending and will never start unless users then manually tell the pipelinerun to start.

Steps to Reproduce the Problem

1. In an environment with Tekton and Tekton concurrency controller configured. Create a ConcurrencyControl resource.
2. Start a pipelinerun in a NS monitored by the concurrency controller and ensuring the pipelinerun has all the required labels to match the created concurrencyControl.
3. Once the pipelinerun is complete in the Tekton dashboard trigger a rerun of that pipelinerun.
4. the newly created pipelinerun will be created in state PipelineRunPending and it will never move forward.

Additional Info

• Kubernetes version:

Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0", GitCommit:"a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2", GitTreeState:"clean", BuildDate:"2022-08-23T17:44:59Z", GoVersion:"go1.19", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.8-eks-ffeb93d", GitCommit:"abb98ec0631dfe573ec5eae40dc48fd8f2017424", GitTreeState:"clean", BuildDate:"2022-11-29T18:45:03Z", GoVersion:"go1.18.8", Compiler:"gc", Platform:"linux/amd64"}

• Tekton Pipeline version:

  Output of tkn version or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'

  Client version: 0.29.1
  Pipeline version: v0.40.2
  Triggers version: v0.20.2
  Dashboard version: v0.29.2

The only error i could find was in the tekton-pipelines-controller logs:

{"severity":"error","timestamp":"2023-02-22T14:52:15.488Z","logger":"tekton-pipelines-controller","caller":"cache/cacheclient.go:58","message":"Unable to fetch client from context.","commit":"40a438a","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"b4f11d5e-65e7-48f6-9ebb-930a2329b1d2","knative.dev/key":"world-pipeline/env-justin-srn6v-r-fs7dv","stacktrace":"github.com/tektoncd/pipeline/pkg/reconciler/events/cache.Get\n\tgithub.com/tektoncd/pipeline/pkg/reconciler/events/cache/cacheclient.go:58\ngithub.com/tektoncd/pipeline/pkg/reconciler/events/cloudevent.SendCloudEventWithRetries\n\tgithub.com/tektoncd/pipeline/pkg/reconciler/events/cloudevent/cloud_event_controller.go:143\ngithub.com/tektoncd/pipeline/pkg/reconciler/events.Emit\n\tgithub.com/tektoncd/pipeline/pkg/reconciler/events/event.go:64\ngithub.com/tektoncd/pipeline/pkg/reconciler/pipelinerun.(*Reconciler).finishReconcileUpdateEmitEvents\n\tgithub.com/tektoncd/pipeline/pkg/reconciler/pipelinerun/pipelinerun.go:284\ngithub.com/tektoncd/pipeline/pkg/reconciler/pipelinerun.(*Reconciler).ReconcileKind\n\tgithub.com/tektoncd/pipeline/pkg/reconciler/pipelinerun/pipelinerun.go:235\ngithub.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/pipelinerun.(*reconcilerImpl).Reconcile\n\tgithub.com/tektoncd/pipeline/pkg/client/injection/reconciler/pipeline/v1beta1/pipelinerun/reconciler.go:235\nknative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/controller/controller.go:542\nknative.dev/pkg/controller.(*Impl).RunContext.func3\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/controller/controller.go:491"}

[tekton-robot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale with a justification. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten with a justification. Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen with a justification. Mark the issue as fresh with /remove-lifecycle rotten with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to [this](https://github.com/tektoncd/experimental/issues/937#issuecomment-1646620633): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen` with a justification. >Mark the issue as fresh with `/remove-lifecycle rotten` with a justification. >If this issue should be exempted, mark the issue as frozen with `/lifecycle frozen` with a justification. > >/close > >Send feedback to [tektoncd/plumbing](https://github.com/tektoncd/plumbing). Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.