WOODPECKER_LETS_ENCRYPT support is crashing the server

philipkozeny commented 1 year ago

Component

server

Describe the bug

I have setup a new instance of woodpecker via docker-compose (see config) below. We are using Forgejo / Gitea and I have setup one Go Repository to setup the process. At the end of the first stage (test) the woodpecker server crashes every time.

System Info

Woodpecker: {"source":"https://github.com/woodpecker-ci/woodpecker","version":"0.15.7"}
Debian 11.6
Docker version 23.0.1, build a5ee5b1

Additional context

docker-compose.yml

version: '3'

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:latest
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./data:/var/lib/woodpecker/
    environment:
      - WOODPECKER_OPEN=true
      - WOODPECKER_HOST=https://url
      - WOODPECKER_AGENT_SECRET=xxx
      - WOODPECKER_GITEA=true
      - WOODPECKER_GITEA_URL=https://url
      - WOODPECKER_GITEA_CLIENT=xxx
      - WOODPECKER_GITEA_SECRET=xxx
      - WOODPECKER_LETS_ENCRYPT=true

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:latest
    command: agent
    restart: always
    depends_on:
      - woodpecker-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WOODPECKER_SERVER=woodpecker-server:9000
      - WOODPECKER_AGENT_SECRET=xxx

volumes:
  woodpecker-server-data:

.woodpecker.yml

pipeline:
  test:
    image: golang
    commands:
        - go fmt $(go list ./... | grep -v /vendor/)
        - go vet $(go list ./... | grep -v /vendor/ | grep -v xy/cli)
        - go vet --structtag=false url/cli
        - go test -race $(go list ./... | grep -v /vendor/)
  build:
    image: golang
    commands:
        - go build -race -ldflags "-extldflags '-static'" -o /framedec

sudo docker logs -f woodpecker_woodpecker-server_1

{"time":"2023-03-24T07:53:52Z","message":"LogLevel = warn"}
{"level":"warn","time":"2023-03-24T07:53:52Z","message":"no sqlite3 file found, will create one at '/var/lib/woodpecker/woodpecker.sqlite'"}
2023/03/24 07:54:00 http: TLS handshake error from 3.15.43.82:57724: tls: client requested unsupported application protocols ([acme-tls/1])
2023/03/24 07:54:01 http: TLS handshake error from 52.38.114.89:22430: tls: client requested unsupported application protocols ([acme-tls/1])
2023/03/24 07:54:01 http: TLS handshake error from 23.178.112.102:54408: tls: client requested unsupported application protocols ([acme-tls/1])
{"level":"warn","error":"sql: no rows in result set","time":"2023-03-24T07:54:15Z"}
{"level":"warn","error":"sql: no rows in result set","time":"2023-03-24T07:54:17Z"}
{"level":"warn","error":"sql: no rows in result set","time":"2023-03-24T07:54:38Z"}
panic: Write called after Handler finished

goroutine 535 [running]:
net/http.(*http2responseWriter).write(0xc0005740c0?, 0xc0008c0e88?, {0x0?, 0xfb9c40?, 0x10521c0?}, {0x1110760?, 0xc000526040?})
    /usr/local/go/src/net/http/h2_bundle.go:6463 +0x150
net/http.(*http2responseWriter).WriteString(0xc0008c0ec0?, {0x1110760?, 0xc000526040?})
    /usr/local/go/src/net/http/h2_bundle.go:6456 +0x2e
io.WriteString({0x7fd1e4569100, 0xc000526040}, {0x1110760, 0x18})
    /usr/local/go/src/io/io.go:313 +0x5d
github.com/gin-gonic/gin.(*responseWriter).WriteString(0xc0007e4300, {0x1110760, 0x18})
    /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/response_writer.go:85 +0x7b
io.WriteString({0x7fd1e4438b98, 0xc0007e4300}, {0x1110760, 0x18})
    /usr/local/go/src/io/io.go:313 +0x5d
github.com/woodpecker-ci/woodpecker/server/api.LogStreamSSE.func2()
    /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/api/stream.go:199 +0x192
created by github.com/woodpecker-ci/woodpecker/server/api.LogStreamSSE
    /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/api/stream.go:179 +0x59b

Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Checked that the bug isn't fixed in the next version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]
[X] Check that this is a concrete bug. For Q&A join our Discord Chat Server or the Matrix room.

philipkozeny commented 1 year ago

This also happens with next btw

philipkozeny commented 1 year ago

FYI: Implementing a proxy in front of Woodpecker (Caddy) and disable the SSL setting is resolving the issue.

karatekaneen commented 2 months ago

This is still a problem in 2.7.1 (latest)

dessalines commented 1 month ago

I'm also getting this frequently.

karatekaneen commented 1 month ago

@dessalines A simple workaround is the one @philipkozeny suggested above. Disable TLS in Woodpecker and put Caddy in front of it and let it handle TLS.

woodpecker-ci / woodpecker