"500 Internal Server Error" when calling GET /go/api/admin/pipelines/:pipeline_name

[adityasood]

Issue Type

• Bug Report

Summary

Getting "500 Internal Server Error" when calling pipelines API for pipelines backed by config repo with YAML config. Seems to be because of missing destination K/V in YAML and JSON.

Environment

Basic environment details

• Go Version: 18.10.0 (7703-42d1cbe661161b5400289ead86c0447c84af8c0a)
• JAVA Version: 1.8.0_171
• OS: Linux 3.10.0-693.2.2.el7.x86_64
• Browser vendor and version (if relevant): curl and node

Additional Environment Details

Steps to Reproduce

1. curl API without destination key, will get HTTP 500 error.
2. Add destination to ci.gocd.yaml file.
3. curl API with destination: "" key works fine.

See config in code snippet below. curl: (22) The requested URL returned error: 500 Server Error

format_version: 3
pipelines:
  TestGoPipelineAPI:
    group: DevOps
    materials:
      mygit:
        git: https://github.com/gocd-contrib/getting-started-repo.git
    stages:
    - Build:
        jobs:
          build:
            elastic_profile_id: "rhel7"
            tasks:
            - exec:
                command: mkdir
                arguments:
                - "surefire-reports"
            artifacts:
            - test:
                source: surefire-reports

Adding destination: "" makes it work fine.

format_version: 3
pipelines:
  TestGoPipelineAPI:
    group: DevOps
    materials:
      mygit:
        git: https://github.com/gocd-contrib/getting-started-repo.git
    stages:
    - Build:
        jobs:
          build:
            elastic_profile_id: "rhel7"
            tasks:
            - exec:
                command: mkdir
                arguments:
                - "surefire-reports"
            artifacts:
            - test:
                source: surefire-reports
                destination: ""

Expected Results

curl --fail --user user:pwd https://gocd-server.example.com/go/api/admin/pipelines/TestGoPipelineAPI -H 'Accept: application/vnd.go.cd.v6+json'
{
  "_links" : {
    "self" : {
...
      "artifacts" : [ {
        "type" : "test",
        "source" : "surefire-reports",
        "destination" : "testoutput"
      } ],
      "properties" : null
    } ]
  } ],
  "tracking_tool" : null,
  "timer" : null
}

Actual Results

curl --fail --user user:pwd https://gocd-server.example.com/go/api/admin/pipelines/TestGoPipelineAPI -H 'Accept: application/vnd.go.cd.v6+json'
curl: (22) The requested URL returned error: 500 Server Error

Possible Fix

Work around is to add: destination: "" key

Log snippets

java.lang.RuntimeException: Try to write null value into configuration! [com.thoughtworks.go.config.BuiltinArtifactConfig.destination]
    at com.thoughtworks.go.util.ExceptionUtils.bomb(ExceptionUtils.java:30)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter$AttributeXmlFieldWithValue.populate(MagicalGoConfigXmlWriter.java:278)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter.write(MagicalGoConfigXmlWriter.java:146)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter.access$200(MagicalGoConfigXmlWriter.java:43)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter$CollectionXmlFieldWithValue.populate(MagicalGoConfigXmlWriter.java:342)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter$ExplicitCollectionXmlFieldWithValue.populate(MagicalGoConfigXmlWriter.java:363)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter.write(MagicalGoConfigXmlWriter.java:146)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter.access$200(MagicalGoConfigXmlWriter.java:43)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter$CollectionXmlFieldWithValue.populate(MagicalGoConfigXmlWriter.java:342)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter$ExplicitCollectionXmlFieldWithValue.populate(MagicalGoConfigXmlWriter.java:363)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter.write(MagicalGoConfigXmlWriter.java:146)
    at com.thoughtworks.go.config.MagicalGoConfigXmlWriter.toXmlPartial(MagicalGoConfigXmlWriter.java:120)
    at com.thoughtworks.go.server.service.EntityHashingService.computeMd5For(EntityHashingService.java:203)
    at com.thoughtworks.go.server.service.EntityHashingService.getFromCache(EntityHashingService.java:180)
    at com.thoughtworks.go.server.service.EntityHashingService.md5ForEntity(EntityHashingService.java:110)
    at com.thoughtworks.go.apiv6.admin.pipelineconfig.PipelineConfigControllerV6.etagFor(PipelineConfigControllerV6.java:74)
    at com.thoughtworks.go.apiv6.admin.pipelineconfig.PipelineConfigControllerV6.etagFor(PipelineConfigControllerV6.java:53)
    at com.thoughtworks.go.api.CrudController.isGetOrHeadRequestFresh(CrudController.java:32)
    at com.thoughtworks.go.apiv6.admin.pipelineconfig.PipelineConfigControllerV6.show(PipelineConfigControllerV6.java:172)
    at spark.RouteImpl$1.handle(RouteImpl.java:72)
    at spark.http.matching.Routes.execute(Routes.java:61)
    at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:130)
    at spark.servlet.SparkFilter.doFilter(SparkFilter.java:173)
    at com.thoughtworks.go.spark.SparkPreFilter.doFilter(SparkPreFilter.java:53)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:420)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:566)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:203)
    at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:73)
    at org.tuckey.web.filters.urlrewrite.NormalRewrittenUrl.doRewrite(NormalRewrittenUrl.java:213)
    at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:171)
    at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
    at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
    at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:381)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
    at com.thoughtworks.go.server.web.FlashLoadingFilter.doFilterInternal(FlashLoadingFilter.java:39)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
    at com.thoughtworks.go.server.newsecurity.filters.DenyIfRefererIsNotFilesFilter.doFilterInternal(DenyIfRefererIsNotFilesFilter.java:53)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
    at com.thoughtworks.go.server.newsecurity.filters.VerifyAuthorityFilter.doFilterInternal(VerifyAuthorityFilter.java:58)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
    at com.thoughtworks.go.server.newsecurity.filters.AbstractUserEnabledCheckFilter.doFilterInternal(AbstractUserEnabledCheckFilter.java:67)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at com.thoughtworks.go.server.newsecurity.filters.ThreadLocalUserFilter.doFilterInternal(ThreadLocalUserFilter.java:42)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
    at com.thoughtworks.go.server.newsecurity.filters.AbstractBasicAuthenticationFilter.filterWhenSecurityEnabled(AbstractBasicAuthenticationFilter.java:99)
    at com.thoughtworks.go.server.newsecurity.filters.AbstractBasicAuthenticationFilter.doFilterInternal(AbstractBasicAuthenticationFilter.java:68)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at com.thoughtworks.go.server.newsecurity.filters.AssumeAnonymousUserFilter.doFilterInternal(AssumeAnonymousUserFilter.java:65)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at com.thoughtworks.go.server.newsecurity.filters.AbstractReAuthenticationFilter.doFilterInternal(AbstractReAuthenticationFilter.java:75)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at com.thoughtworks.go.server.newsecurity.filters.InvalidateAuthenticationOnSecurityConfigChangeFilter.doFilterInternal(InvalidateAuthenticationOnSecurityConfigChangeFilter.java:84)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
    at com.thoughtworks.go.server.newsecurity.filters.AlwaysCreateSessionFilter.doFilterInternal(AlwaysCreateSessionFilter.java:41)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at com.thoughtworks.go.server.newsecurity.filters.AbstractSessionReduceIdleTimeoutFilter.doFilterInternal(AbstractSessionReduceIdleTimeoutFilter.java:45)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:185)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at com.thoughtworks.go.server.newsecurity.filters.ModeAwareFilter.doFilter(ModeAwareFilter.java:53)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
    at com.thoughtworks.go.server.newsecurity.filterchains.MainFilterChain.doFilter(MainFilterChain.java:76)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at com.thoughtworks.go.server.web.BackupFilter.doFilter(BackupFilter.java:88)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at com.thoughtworks.go.server.web.DefaultHeadersFilter.doFilter(DefaultHeadersFilter.java:36)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.Server.handle(Server.java:530)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
    at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:289)
    at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:149)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
    at java.lang.Thread.run(Thread.java:748)

Code snippets/Screenshots

Any other info

[adityasood]

/cc @chrislovsund

[tomzo]

There was a similar problem with environment variables being not set vs "". I think the right place to fix it is the ConfigConverter in gocd server because it will deal with all plugins returning null value. See how I fixed https://github.com/gocd/gocd/pull/4696

[adityasood]

@tomzo yes you are right this is to be done at the server side. closing this issue.