tektoncd / pipeline

A cloud-native Pipeline resource.
https://tekton.dev
Apache License 2.0
8.51k stars 1.78k forks source link

Metrics Exporter Bundles Remote Tasks Together and Labels Them as "Anonymous" #8215

Open OsamaKhan220 opened 3 months ago

OsamaKhan220 commented 3 months ago

Description:

When running a pipeline with a mix of ClusterTask and remote Task definitions, the Tekton controller correctly exports metrics for the ClusterTask. However, metrics for the remote tasks are bundled together and labeled as "anonymous" in Prometheus.

Steps to Reproduce:

  1. Define a pipeline with both ClusterTask and remote Task references.
  2. Execute the pipeline.
  3. Observe the metrics exported by the Tekton controller.

Expected Behavior: Each task, whether ClusterTask or remote Task, should have its own distinct metrics exported, labeled with the appropriate task name (e.g., git-clone or stakater-create-git-tag).

Actual Behavior: Metrics for remote tasks are bundled together and labeled as "anonymous," making it difficult to distinguish between them.

Example Pipeline Snippet:

Copy code
- name: fetch-repository
  taskRef:
    name: git-clone
    kind: ClusterTask
  params:
    - name: depth
      value: "0"

- name: create-git-tag
  taskRef:
    name: stakater-create-git-tag
    kind: Task

- name: create-environment
  taskRef:
    kind: Task
    name: stakater-create-environment
  params:
chengjoey commented 3 months ago

+1, it seems that not only clusterTask or remoteTask, but also ordinary tasks will have anonymous

tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="900"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="1800"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="3600"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="5400"} 6
100 88827    0 88827    0     0  18.4M      0 --:--:-- --:--:-- --:--:-- 21.1M
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="10800"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="21600"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="43200"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="86400"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="+Inf"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_sum{namespace="default",pipeline="volume-from-template",status="success",task="anonymous"} 57.044900693
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_count{namespace="default",pipeline="volume-from-template",status="success",task="anonymous"} 6
tekton_pipelines_controller_taskruns_pod_latency_milliseconds{namespace="default",pod="run-with-template-reader-pod",task="anonymous"} 0
tekton_pipelines_controller_taskruns_pod_latency_milliseconds{namespace="default",pod="run-with-template-writer-pod",task="anonymous"} 1000
tekton_pipelines_controller_taskruns_pod_latency_milliseconds{namespace="default",pod="run-with-template-xmdls-reader-pod",task="anonymous"} 0
tekton_pipelines_controller_taskruns_pod_latency_milliseconds{namespace="default",pod="run-with-template-xmdls-writer-pod",task="anonymous"} 1000
chengjoey commented 3 months ago
  1. cluster task use different key https://github.com/tektoncd/pipeline/blob/62ae6f006c30fdd055f007cb9e89ec10495dd9bb/pkg/reconciler/taskrun/taskrun.go#L1055-L1062

  2. taskrun.Spec.TaskSpec != nil, it may be created by pipelinerun, use tekton.dev/pipelineTask

    kubectl describe taskrun run-with-template-writer
    Name:         run-with-template-writer
    Namespace:    default
    Labels:       app.kubernetes.io/managed-by=tekton-pipelines
              tekton.dev/memberOf=tasks
              tekton.dev/pipeline=volume-from-template
              tekton.dev/pipelineRun=run-with-template
              tekton.dev/pipelineTask=writer
chengjoey commented 2 days ago

/close