search

tcplugins / tcWebHooks

WebHooks plugin for Teamcity. Supports many build states and payload formats.
https://netwolfuk.wordpress.com/category/teamcity/tcplugins/tcwebhooks/
155 stars 30 forks source link

Unexpected Error when accessing webhook search results #229

Closed david-mcfall-fxs closed 3 months ago

david-mcfall-fxs commented 10 months ago

Expected Behavior

Webhook search results should be displayed

Current Behavior

When we click any of the links that use webhooks/search.html, TeamCity displays an unexpected error and a stack trace.

Steps to Reproduce (for bugs)

  1. In TeamCity, navigate to Administration/WebHooks
  2. Where it says: "There are ## WebHooks configured in this TeamCity Server.", click the "## WebHooks" link.

Your Environment

netwolfuk commented 10 months ago

I'm not able to reproduce this on 1.2.5 and there is no code in the diff from 1.2.4 to 1.2.5 that would affect this. Can I get more detail on the stack trace? There will be a stack trace on the user's page, and also in the teamcity-server.log

david-mcfall-fxs commented 10 months ago

Trace below:

Trace: java.lang.NullPointerException at webhook.teamcity.extension.bean.WebhookConfigAndBuildTypeListHolder.(WebhookConfigAndBuildTypeListHolder.java:90) at webhook.teamcity.extension.bean.ProjectWebHooksBean.addWebHookConfigHolder(ProjectWebHooksBean.java:168) at webhook.teamcity.extension.bean.ProjectWebHooksBean.buildWithoutNew(ProjectWebHooksBean.java:144) at webhook.teamcity.extension.WebHookSearchController.doHandle(WebHookSearchController.java:78) at jetbrains.buildServer.controllers.BaseController.handleRequestInternal(BaseController.java:114) at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:177) at jetbrains.buildServer.controllers.BaseController.handleRequest(BaseController.java:93) at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:51) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963) at jetbrains.buildServer.maintenance.WebDispatcherServlet.doService(WebDispatcherServlet.java:38) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:529) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:623) at jetbrains.buildServer.maintenance.TeamCityDispatcherServlet.processedByMainServlet(TeamCityDispatcherServlet.java:8) at jetbrains.buildServer.maintenance.TeamCityDispatcherServlet.service(TeamCityDispatcherServlet.java:43) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at jetbrains.buildServer.https.HttpsRedirectFilter.doFilter(HttpsRedirectFilter.java:31) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at jetbrains.buildServer.web.jsp.JspPrecompilerFilter.doFilter(JspPrecompilerFilter.java:181) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at jetbrains.buildServer.web.DisableSessionIdFromUrlFilter.doFilter(DisableSessionIdFromUrlFilter.java:8) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at jetbrains.buildServer.web.UserIdProviderFilter.doFilter(UserIdProviderFilter.java:7) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at jetbrains.buildServer.web.BannedIPsFilter.doFilter(BannedIPsFilter.java:3) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at jetbrains.buildServer.web.NodeInfoHeaderFilter.doFilter(NodeInfoHeaderFilter.java:7) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108) at jetbrains.buildServer.diagnostic.web.DiagnosticFilter.doFilter(DiagnosticFilter.java:2) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at jetbrains.buildServer.web.DependencyParametersCalculationContextFilter.doFilter(DependencyParametersCalculationContextFilter.java:10) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at jetbrains.buildServer.diagnostic.web.HttpRequestsDurationMetricsReporter.doFilter(HttpRequestsDurationMetricsReporter.java:4) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at jetbrains.buildServer.web.HttpSecurityHeadersFilter.doFilter(HttpSecurityHeadersFilter.java:58) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at jetbrains.buildServer.controllers.filters.DisableSessionCookieTokenAuthFilter.doFilter(DisableSessionCookieTokenAuthFilter.java:11) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at jetbrains.buildServer.controllers.filters.ProxyDetailsFilter.doFilter(ProxyDetailsFilter.java:11) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at jetbrains.buildServer.controllers.filters.ClearSecurityContextFilter.doFilter(ClearSecurityContextFilter.java:15) at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) at org.springframework.web.filter.CompositeFilter.doFilter(CompositeFilter.java:74) at jetbrains.buildServer.web.DelegatingFilter.doFilter(DelegatingFilter.java:74) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at jetbrains.buildServer.web.ResponseFragmentFilter.doFilter(ResponseFragmentFilter.java:29) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:926) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829)

david-mcfall-fxs commented 10 months ago

Dev has a hunch that it's related to a webhook that was deleted via kotlin script. Perhaps there's a broken relationship causing the crash.

netwolfuk commented 10 months ago

Yes, I am starting to think the same thing. There is a cache that backs the search page, and it has obviously not updated from the edit. I am not sure if I am reacting to the kotlin edit broadcast event.

It might be possible to just edit a webhook in the same project using the webhook UI. That should refresh the cache, but I'll need to check the code. It's worth a try for now.

netwolfuk commented 10 months ago

I found a project on my local teamcity that has VCS config setup. Although, mine is XML, not Kotlin.
I removed a webhook and can still see it in the list. However, trying to edit it or delete it fails.

I then edited another webhook in the UI for the same project, but the cache is only updated for that webhook, not refreshed for all the webhooks in that project.

I can't however reproduce the exact same issue you are having with the search page.

Actions for me:

  1. Create a kotlin config and do more testing.
  2. Provide a way to refresh the cache. This might be easiest done by refreshing the cache for all webhooks for a project on every edit.
  3. Figure out how to receive events when config is updated via config as code (kotlin or XML), and refresh the cache then.

If you have some more detail about what the devs did to get the stack trace that would be good.

david-mcfall-fxs commented 10 months ago

Some actions recalled from dev: "I had multiple tabs open of the project, and I deleted the webhook from this sub-project and another sub-project, I moved a template from the sub-project to root, I added webhooks on the root project, then I went to that sub-project to move the template it was using to root."

netwolfuk commented 10 months ago

Thanks @david-mcfall-fxs for the updated information.

I've converted my Versioned Settings to kotlin. With further testing like the above I still can't reproduce the issue sorry.

For you, if you retart TeamCity, this will fix the issue. The cache is only held in memory, and restarting will call the initialise function to repopulate the cache.

I have raised a support request with Jetbrains to ask how to listen for events when the configuration is edited via VCS.

I have also coded up a fix that rebuilds the cache for all webhooks on a project when a webhook on that project is edited. I will test that and then do a release (assuming I don't get reply from Jetbrains) ASAP. However, this week is very busy already.

Your best next step is to restart TeamCity. This will at least force the cache to rebuild, and should fix the error you're seeing.

david-mcfall-fxs commented 10 months ago

No worries, TC service restarts are not terribly disruptive for us. I restarted the service a few minutes ago and we have the webhook searches working again. Thank you for all your efforts on this.

netwolfuk commented 3 months ago

released a version that will fix this issue: https://github.com/tcplugins/tcWebHooks/releases/tag/v2.0.1