woodpecker-ci / woodpecker

Woodpecker is a simple, yet powerful CI/CD engine with great extensibility.
https://woodpecker-ci.org
Apache License 2.0
4.25k stars 369 forks source link

CI no longer starts #1609

Closed NextFire closed 1 year ago

NextFire commented 1 year ago

Component

server

Describe the bug

On one of my projects CI is still triggered on push events but pipelines do not start anymore.

There is a runtime error on the server just after the push. Some other warnings complain about secrets not found but they are well defined at the organization level and the other project CIs are working fine.

Also, I can't cancel these stalled pipelines. I have already tried to reinstall Woodpecker without success.

System Info

{"source":"https://github.com/woodpecker-ci/woodpecker","version":"next-d35ab57f"}

Deployed on Kubernetes
Helmfile: https://github.com/Japan7/nanak8s/blob/main/apps/woodpecker/helmfile.yaml
Server chart: https://github.com/babykart/woodpecker-helm/tree/server-sts/woodpecker-server
Agent chart: https://github.com/woodpecker-ci/helm/tree/main/woodpecker-agent

Additional context

[woodpecker-server-0] [xorm] [info]  2023/03/09 01:21:37.732964 PING DATABASE sqlite3
[woodpecker-server-0] [xorm] [info]  2023/03/09 01:21:37.732964 PING DATABASE sqlite3
[woodpecker-server-0] {"level":"error","error":"no secret found for \"registry_username\"","time":"2023-03-09T01:21:42Z","message":"paramsToEnv"}
[woodpecker-server-0] {"level":"error","error":"no secret found for \"registry_username\"","time":"2023-03-09T01:21:42Z","message":"paramsToEnv"}
[woodpecker-server-0] {"level":"error","error":"no secret found for \"webhook_id\"","time":"2023-03-09T01:21:42Z","message":"paramsToEnv"}
[woodpecker-server-0] {"level":"error","error":"no secret found for \"webhook_token\"","time":"2023-03-09T01:21:42Z","message":"paramsToEnv"}
[woodpecker-server-0] {"level":"error","error":"no secret found for \"webhook_id\"","time":"2023-03-09T01:21:42Z","message":"paramsToEnv"}
[woodpecker-server-0] 
[woodpecker-server-0] 
[woodpecker-server-0] 2023/03/09 01:21:47 [Recovery] 2023/03/09 - 01:21:47 panic recovered:
[woodpecker-server-0] runtime error: invalid memory address or nil pointer dereference
[woodpecker-server-0] /usr/local/go/src/runtime/panic.go:260 (0x454e95)
[woodpecker-server-0] /usr/local/go/src/runtime/signal_unix.go:835 (0x454e65)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/pipeline/frontend/yaml/compiler/settings/params.go:115 (0xc0b519)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/pipeline/frontend/yaml/compiler/settings/params.go:37 (0xc0ac16)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/pipeline/frontend/yaml/compiler/convert.go:82 (0xc151cd)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/pipeline/frontend/yaml/compiler/compiler.go:244 (0xc13de4)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/pipeline/stepBuilder.go:290 (0xc19f95)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/pipeline/stepBuilder.go:140 (0xc185d2)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/pipeline/items.go:81 (0xc373a4)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/pipeline/create.go:133 (0xc359b7)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/api/hook.go:178 (0xc7d198)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xccfe8d)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/token/token.go:33 (0xccfa18)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xc74fbc)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/session/user.go:72 (0xc74fa3)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xcd523e)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/store.go:29 (0xcd5225)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xcd4ae7)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/logger.go:23 (0xcd4aca)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xccf879)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/header/header.go:38 (0xccf7a4)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xccf701)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/middleware/header/header.go:30 (0xccf6e6)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xcd3eca)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/router/router.go:39 (0xcd3eb0)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xa289e1)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/recovery.go:101 (0xa289cc)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/context.go:173 (0xa27850)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/gin.go:616 (0xa274b8)
[woodpecker-server-0] /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/gin.go:572 (0xa26ffc)
[woodpecker-server-0] /usr/local/go/src/net/http/server.go:2947 (0x79510b)
[woodpecker-server-0] /usr/local/go/src/net/http/server.go:1991 (0x790326)
[woodpecker-server-0] /usr/local/go/src/runtime/asm_amd64.s:1594 (0x472920)
[woodpecker-server-0] 
[woodpecker-server-0] [xorm] [info]  2023/03/09 01:21:47.732848 PING DATABASE sqlite3
[woodpecker-server-0] [xorm] [info]  2023/03/09 01:21:47.732873 PING DATABASE sqlite3

https://woodpecker.japan7.bde.enseeiht.fr/Japan7/nanapi/pipeline/1

Capture d’écran 2023-03-08 à 20 37 59

Validations

NextFire commented 1 year ago

Alright, I found the culprit, it was my mistake: both docker-push and docker-tag jobs had when.event: push (the later should have when.event: tag). Maybe there should be a more explicit error when this kind of thing happens :/