Completed jobs still hanging in the UI

samuelvgo commented 3 weeks ago

Issue Type

Bug Report

Summary

Pipelines that aren't triggered via the API are complete but still showing as hanging in the UI

Environment

Basic environment details

Go Version: 23.5.0 (18179-7702b283accd1f90f014f0087aa2e9bd8baf4a97)
JAVA Version: 17.0.9
OS: Linux 4.19.0-23-cloud-amd64

Additional Environment Details

Steps to Reproduce

Expected Results

I would expect the pipeline to report completed and not as not responding in the UI

Actual Results

Pipeline is completed and successful but reported as not responding in the UI

Possible Fix

Log snippets

2024-04-22 12:37:51,973 INFO  [qtp1871368707-30] BuildRepositoryRemoteImpl:96 - [Agent [opglive-europe-west4-gocd-agent-0, 10.100.32.3, c85bfce9-1733-4613-b678-a9c27665ce8e, da11a799-01f6-4a86-940b-ea0798d49e59]] is reporting status [Completing] for [Build [tech-docs/146/opglive-europe-west4/1/deploy-docker/52112]]
2024-04-22 12:37:52,247 INFO  [qtp1871368707-30] BuildRepositoryRemoteImpl:96 - [Agent [opglive-europe-west4-gocd-agent-0, 10.100.32.3, c85bfce9-1733-4613-b678-a9c27665ce8e, da11a799-01f6-4a86-940b-ea0798d49e59]] is reporting status and result [Completed, Passed] for [Build [tech-docs/146/opglive-europe-west4/1/deploy-docker/52112]]
2024-04-22 12:37:52,261 INFO  [qtp1871368707-30] Stage:236 - Stage is being completed by transition id: 308480
2024-04-22 12:37:52,286 ERROR [102@MessageListener for JobStatusListener] JMSMessageListenerAdapter:87 - Exception thrown in message handling by listener com.thoughtworks.go.server.messaging.JobStatusListener@6aea2145
java.lang.RuntimeException: Does not contain plan with name deploy-docker
        at com.thoughtworks.go.util.ExceptionUtils.bomb(ExceptionUtils.java:26)
        at com.thoughtworks.go.domain.JobInstances.getByName(JobInstances.java:46)
        at com.thoughtworks.go.domain.Stage.findJob(Stage.java:273)
        at com.thoughtworks.go.server.messaging.JobStatusListener.onMessage(JobStatusListener.java:71)
        at com.thoughtworks.go.server.messaging.JobStatusListener.onMessage(JobStatusListener.java:30)
        at com.thoughtworks.go.server.messaging.activemq.JMSMessageListenerAdapter.runImpl(JMSMessageListenerAdapter.java:83)
        at com.thoughtworks.go.server.messaging.activemq.JMSMessageListenerAdapter.run(JMSMessageListenerAdapter.java:63)
        at java.base/java.lang.Thread.run(Unknown Source)

 2024-04-22 12:33:24,184 INFO  [qtp1871368707-42] BuildRepositoryRemoteImpl:96 - [Agent [sandbox-europe-west4-nomad-server-1, 172.20.0.3, 990b97ac-e9ed-4ad2-b51e-b94ec90662ab, bb712a22-9d70-441c-90e7-20bcb3068e98]] is reporting result [Passed] for [Build [consul-alerts-nomad/12/sandbox-europe-west4/7/nomad/52111]]
2024-04-22 12:33:24,236 INFO  [qtp1871368707-33] BuildRepositoryRemoteImpl:96 - [Agent [sandbox-europe-west4-nomad-server-1, 172.20.0.3, 990b97ac-e9ed-4ad2-b51e-b94ec90662ab, bb712a22-9d70-441c-90e7-20bcb3068e98]] is reporting status [Completing] for [Build [consul-alerts-nomad/12/sandbox-europe-west4/7/nomad/52111]]
2024-04-22 12:33:24,281 INFO  [qtp1871368707-33] BuildRepositoryRemoteImpl:96 - [Agent [sandbox-europe-west4-nomad-server-1, 172.20.0.3, 990b97ac-e9ed-4ad2-b51e-b94ec90662ab, bb712a22-9d70-441c-90e7-20bcb3068e98]] is reporting status and result [Completed, Passed] for [Build [consul-alerts-nomad/12/sandbox-europe-west4/7/nomad/52111]]
2024-04-22 12:33:24,294 INFO  [qtp1871368707-33] Stage:236 - Stage is being completed by transition id: 308474
2024-04-22 12:33:24,314 ERROR [102@MessageListener for JobStatusListener] JMSMessageListenerAdapter:87 - Exception thrown in message handling by listener com.thoughtworks.go.server.messaging.JobStatusListener@6aea2145
java.lang.RuntimeException: Does not contain plan with name nomad
        at com.thoughtworks.go.util.ExceptionUtils.bomb(ExceptionUtils.java:26)
        at com.thoughtworks.go.domain.JobInstances.getByName(JobInstances.java:46)
        at com.thoughtworks.go.domain.Stage.findJob(Stage.java:273)
        at com.thoughtworks.go.server.messaging.JobStatusListener.onMessage(JobStatusListener.java:71)
        at com.thoughtworks.go.server.messaging.JobStatusListener.onMessage(JobStatusListener.java:30)
        at com.thoughtworks.go.server.messaging.activemq.JMSMessageListenerAdapter.runImpl(JMSMessageListenerAdapter.java:83)
        at com.thoughtworks.go.server.messaging.activemq.JMSMessageListenerAdapter.run(JMSMessageListenerAdapter.java:63)
        at java.base/java.lang.Thread.run(Unknown Source)

Code snippets/Screenshots

Any other info

chadlwilson commented 3 weeks ago

Similar to #12762 you need to include a precise set of steps that you are performing that creates this problem. You haven’t followed up there with the sequence of events that you are triggering so I’m not so confident we are going to get anywhere here either.

There’s not much value in creating new issues if you’re not following up with the requested information on the ones you already created.

This error implies something very wrong is happening, such as deleting dependencies or parts of pipelines while they are still running, or something like that. Please share more details on how to reproduce the error or more details in what you are doing with these pipelines, or there’s likely nothing that can be done here.

samuelvgo commented 3 weeks ago

Sorry, missed the steps to reproduce.

All of these pipelines were not triggered via the API, they are manually triggered in the UI (eg.: consul-alerts) or triggered by a change in the git repository (eg: tech-docs). A couple of these pipelines have multiple tasks, around 4, but I don't see how that could be a problem here.

These hanging errors weren't happening with the 19.12v. Please let me know if more details are needed, there is nothing special with these pipelines other than what I mentioned.

gocd / gocd