goauthentik / authentik

The authentication glue you need.
https://goauthentik.io
Other
13.58k stars 909 forks source link

proxy-outpost crashes with "panic: repeated read on failed websocket connection" #11090

Open rlenferink opened 2 months ago

rlenferink commented 2 months ago

Describe the bug The proxy outpost just crashed for me:

{"error":"websocket: close 1012","event":"ws read error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-handler","timestamp":"2024-08-25T20:53:23Z"}
{"error":"websocket: close 1012","event":"ws read error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-handler","timestamp":"2024-08-25T20:53:28Z"}
{"error":"websocket: close sent","event":"ws write error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-health","timestamp":"2024-08-25T20:53:33Z"}
{"error":"websocket: close 1012","event":"ws read error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-handler","timestamp":"2024-08-25T20:53:33Z"}
{"error":"websocket: close 1012","event":"ws read error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-handler","timestamp":"2024-08-25T20:53:38Z"}
{"error":"websocket: close sent","event":"ws write error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-health","timestamp":"2024-08-25T20:53:43Z"}
{"error":"websocket: close 1012","event":"ws read error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-handler","timestamp":"2024-08-25T20:53:43Z"}
{"error":"websocket: close 1012","event":"ws read error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-handler","timestamp":"2024-08-25T20:53:48Z"}
{"error":"websocket: close sent","event":"ws write error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-health","timestamp":"2024-08-25T20:53:53Z"}
{"error":"websocket: close 1012","event":"ws read error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-handler","timestamp":"2024-08-25T20:53:53Z"}
panic: repeated read on failed websocket connection

goroutine 38 [running]:
github.com/gorilla/websocket.(*Conn).NextReader(0xc00028c6e0)
    /go/pkg/mod/github.com/gorilla/websocket@v1.5.3/conn.go:1030 +0x1fb
github.com/gorilla/websocket.(*Conn).ReadJSON(0x12a05f200?, {0x1263c20, 0xc0001962d0})
    /go/pkg/mod/github.com/gorilla/websocket@v1.5.3/json.go:50 +0x25
goauthentik.io/internal/outpost/ak.(*APIController).startWSHandler(0xc000344a80)
    /go/src/goauthentik.io/internal/outpost/ak/api_ws.go:119 +0x186
goauthentik.io/internal/outpost/ak.(*APIController).StartBackgroundTasks.func1()
    /go/src/goauthentik.io/internal/outpost/ak/api.go:214 +0x5d
created by goauthentik.io/internal/outpost/ak.(*APIController).StartBackgroundTasks in goroutine 1
    /go/src/goauthentik.io/internal/outpost/ak/api.go:212 +0x309

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior The outpost to not crash, report an error and keep on running.

Screenshots If applicable, add screenshots to help explain your problem.

Logs Output of docker-compose logs or kubectl logs respectively

Version and Deployment (please complete the following information):

Additional context Add any other context about the problem here.

rlenferink commented 2 months ago

Apparently next to the repeated read on failed websocket connection the following also occurs:

{"event":"Update providers","level":"info","logger":"authentik.outpost.ldap","timestamp":"2024-09-13T04:33:03Z"}
{"event":"initialised direct binder","level":"info","logger":"authentik.outpost.ldap.binder.direct","timestamp":"2024-09-13T04:38:03Z"}
{"event":"Update providers","level":"info","logger":"authentik.outpost.ldap","timestamp":"2024-09-13T04:38:03Z"}
{"error":"websocket: close 1012","event":"ws read error","level":"warning","logger":"authentik.outpost.ak-api-controller","loop":"ws-handler","timestamp":"2024-09-13T04:39:25Z"}
panic: concurrent write to websocket connection

goroutine 46 [running]:
github.com/gorilla/websocket.(*messageWriter).flushFrame(0xc000473b00, 0x1, {0x0?, 0x0?, 0x0?})
    /go/pkg/mod/github.com/gorilla/websocket@v1.5.3/conn.go:617 +0x4b8
github.com/gorilla/websocket.(*messageWriter).Close(0xc0004bb870?)
    /go/pkg/mod/github.com/gorilla/websocket@v1.5.3/conn.go:731 +0x35
github.com/gorilla/websocket.(*Conn).WriteJSON(0x11f5680?, {0x11f5680, 0xc000215750})
    /go/pkg/mod/github.com/gorilla/websocket@v1.5.3/json.go:29 +0xd6
goauthentik.io/internal/outpost/ak.(*APIController).SendWSHello(0xc00037d500, 0xc0004bbf40)
    /go/src/goauthentik.io/internal/outpost/ak/api_ws.go:230 +0x134
goauthentik.io/internal/outpost/ak.(*APIController).startWSHealth(0xc00037d500)
    /go/src/goauthentik.io/internal/outpost/ak/api_ws.go:167 +0xe5
goauthentik.io/internal/outpost/ak.(*APIController).StartBackgroundTasks.func2()
    /go/src/goauthentik.io/internal/outpost/ak/api.go:218 +0x5d
created by goauthentik.io/internal/outpost/ak.(*APIController).StartBackgroundTasks in goroutine 1
    /go/src/goauthentik.io/internal/outpost/ak/api.go:216 +0x34b
authentik-automation[bot] commented 1 hour ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.