search

openshift-pipelines / pipeline-service

SaaS for Tekton Pipelines
Apache License 2.0
23 stars 44 forks source link

PLNSRVCE-1526: add affinity policies to results watcher #942

Closed gabemontero closed 6 months ago

gabemontero commented 6 months ago

Discovered the need for this as a result of https://github.com/redhat-appstudio/infra-deployments/pull/3252#issuecomment-1937835777

Short term, the situation with the bitnami images and postgresql will sabatoge CI, but a manual apply -k of yaml shows the deployment is created fine and the watcher pod is started successfully

@openshift-pipelines/pipelines-service FYI

rh-pre-commit.version: 2.1.0 rh-pre-commit.check-secrets: ENABLED

ramessesii2 commented 6 months ago

/retest

gabemontero commented 6 months ago

temp bump of watcher to 2 replicas looks good doing dev_setup.sh of this PR's branch:

gmontero ~/go/src/github.com/openshift-pipelines/pipeline-service  (results-watcher-affinity)$ oc get deployment
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
tekton-results-api       1/1     1            1           25m
tekton-results-watcher   2/2     2            2           22m
gmontero ~/go/src/github.com/openshift-pipelines/pipeline-service  (results-watcher-affinity)$ oc get pods 
NAME                                      READY   STATUS    RESTARTS   AGE
postgres-postgresql-0                     1/1     Running   0          25m
storage-pool-0-0                          2/2     Running   0          24m
tekton-results-api-596c5f45fb-wp7xm       2/2     Running   0          25m
tekton-results-watcher-5759c9d6f5-4m4px   2/2     Running   0          35s
tekton-results-watcher-5759c9d6f5-qksvd   2/2     Running   0          22m
gmontero ~/go/src/github.com/openshift-pipelines/pipeline-service  (results-watcher-affinity)$ oc logs tekton-results-watcher-5759c9d6f5-4m4px -c watcher
2024/02/13 15:35:03 dialing tekton-results-api-service.tekton-results.svc.cluster.local:8080...
2024/02/13 15:35:03 Registering 3 clients
2024/02/13 15:35:03 Registering 2 informer factories
2024/02/13 15:35:03 Registering 2 informers
2024/02/13 15:35:03 Registering 2 controllers
{"level":"info","ts":"2024-02-13T15:35:03.685Z","caller":"logging/config.go:80","msg":"Unable to read vcs.revision from binary"}
{"level":"info","ts":"2024-02-13T15:35:03.685Z","logger":"watcher","caller":"profiling/server.go:64","msg":"Profiling enabled: false"}
{"level":"info","ts":"2024-02-13T15:35:03.691Z","logger":"watcher","caller":"leaderelection/context.go:47","msg":"Running with Standard leader election"}
{"level":"info","ts":"2024-02-13T15:35:03.699Z","logger":"watcher","caller":"sharedmain/main.go:283","msg":"Starting configuration manager..."}
{"level":"info","ts":1707838503.79978,"logger":"fallback","caller":"injection/injection.go:63","msg":"Starting informers..."}
{"level":"info","ts":"2024-02-13T15:35:03.900Z","logger":"watcher","caller":"sharedmain/main.go:311","msg":"Starting controllers..."}
{"level":"info","ts":"2024-02-13T15:35:03.900Z","logger":"watcher","caller":"leaderelection/context.go:149","msg":"watcher.taskruns.00-of-01 will run in leader-elected mode with id \"tekton-results-watcher-5759c9d6f5-4m4px_90c79320-b4b9-48b4-85c5-f2d6f0c87a60\""}
{"level":"info","ts":"2024-02-13T15:35:03.900Z","logger":"watcher","caller":"controller/controller.go:486","msg":"Starting controller and workers"}
{"level":"info","ts":"2024-02-13T15:35:03.900Z","logger":"watcher","caller":"controller/controller.go:496","msg":"Started workers"}
{"level":"info","ts":"2024-02-13T15:35:03.900Z","logger":"watcher","caller":"leaderelection/context.go:149","msg":"watcher.pipelineruns.00-of-01 will run in leader-elected mode with id \"tekton-results-watcher-5759c9d6f5-4m4px_1f4c3cca-bdb6-4d3c-bb39-95a6e0bb639b\""}
{"level":"info","ts":"2024-02-13T15:35:03.900Z","logger":"watcher","caller":"controller/controller.go:486","msg":"Starting controller and workers"}
{"level":"info","ts":"2024-02-13T15:35:03.900Z","logger":"watcher","caller":"controller/controller.go:496","msg":"Started workers"}
I0213 15:35:03.900948       1 leaderelection.go:248] attempting to acquire leader lease tekton-results/watcher.taskruns.00-of-01...
I0213 15:35:03.901104       1 leaderelection.go:248] attempting to acquire leader lease tekton-results/watcher.pipelineruns.00-of-01...
gmontero ~/go/src/github.com/openshift-pipelines/pipeline-service  (results-watcher-affinity)$ oc logs tekton-results-watcher-5759c9d6f5-qksvd -c watcher
2024/02/13 15:13:13 dialing tekton-results-api-service.tekton-results.svc.cluster.local:8080...
2024/02/13 15:13:13 Registering 3 clients
2024/02/13 15:13:13 Registering 2 informer factories
2024/02/13 15:13:13 Registering 2 informers
2024/02/13 15:13:13 Registering 2 controllers
{"level":"info","ts":"2024-02-13T15:13:13.287Z","caller":"logging/config.go:80","msg":"Unable to read vcs.revision from binary"}
{"level":"info","ts":"2024-02-13T15:13:13.287Z","logger":"watcher","caller":"profiling/server.go:64","msg":"Profiling enabled: false"}
{"level":"info","ts":"2024-02-13T15:13:13.289Z","logger":"watcher","caller":"leaderelection/context.go:47","msg":"Running with Standard leader election"}
{"level":"info","ts":"2024-02-13T15:13:13.351Z","logger":"watcher","caller":"sharedmain/main.go:283","msg":"Starting configuration manager..."}
{"level":"info","ts":1707837193.45243,"logger":"fallback","caller":"injection/injection.go:63","msg":"Starting informers..."}
{"level":"info","ts":"2024-02-13T15:13:13.553Z","logger":"watcher","caller":"sharedmain/main.go:311","msg":"Starting controllers..."}
{"level":"info","ts":"2024-02-13T15:13:13.553Z","logger":"watcher","caller":"leaderelection/context.go:149","msg":"watcher.taskruns.00-of-01 will run in leader-elected mode with id \"tekton-results-watcher-5759c9d6f5-qksvd_3b1dc059-d858-4a68-92a9-8f0b082eab44\""}
{"level":"info","ts":"2024-02-13T15:13:13.553Z","logger":"watcher","caller":"controller/controller.go:486","msg":"Starting controller and workers"}
{"level":"info","ts":"2024-02-13T15:13:13.553Z","logger":"watcher","caller":"controller/controller.go:496","msg":"Started workers"}
{"level":"info","ts":"2024-02-13T15:13:13.553Z","logger":"watcher","caller":"leaderelection/context.go:149","msg":"watcher.pipelineruns.00-of-01 will run in leader-elected mode with id \"tekton-results-watcher-5759c9d6f5-qksvd_8c539727-7e83-4a43-adf2-dd90a6760b05\""}
{"level":"info","ts":"2024-02-13T15:13:13.553Z","logger":"watcher","caller":"controller/controller.go:486","msg":"Starting controller and workers"}
{"level":"info","ts":"2024-02-13T15:13:13.553Z","logger":"watcher","caller":"controller/controller.go:496","msg":"Started workers"}
I0213 15:13:13.553699       1 leaderelection.go:248] attempting to acquire leader lease tekton-results/watcher.taskruns.00-of-01...
I0213 15:13:13.553874       1 leaderelection.go:248] attempting to acquire leader lease tekton-results/watcher.pipelineruns.00-of-01...
I0213 15:13:13.559913       1 leaderelection.go:258] successfully acquired lease tekton-results/watcher.pipelineruns.00-of-01
{"level":"info","ts":"2024-02-13T15:13:13.560Z","logger":"watcher","caller":"leaderelection/context.go:158","msg":"\"tekton-results-watcher-5759c9d6f5-qksvd_8c539727-7e83-4a43-adf2-dd90a6760b05\" has started leading \"watcher.pipelineruns.00-of-01\""}
I0213 15:13:13.562750       1 leaderelection.go:258] successfully acquired lease tekton-results/watcher.taskruns.00-of-01
{"level":"info","ts":"2024-02-13T15:13:13.562Z","logger":"watcher","caller":"leaderelection/context.go:158","msg":"\"tekton-results-watcher-5759c9d6f5-qksvd_3b1dc059-d858-4a68-92a9-8f0b082eab44\" has started leading \"watcher.taskruns.00-of-01\""}
gmontero ~/go/src/github.com/openshift-pipelines/pipeline-service  (results-watcher-affinity)$
gabemontero commented 6 months ago

only this known results error on the upgrade, which is being seen across multiple PRs, is failing

gabemontero commented 6 months ago

/retest