search

binwiederhier / ntfy

Send push notifications to your phone or desktop using PUT/POST
https://ntfy.sh
Apache License 2.0
17.77k stars 691 forks source link

Thousands ad thousands of defunct ssl client orphans... #956

Closed emigrating closed 9 months ago

emigrating commented 9 months ago

:lady_beetle: Describe the bug

Not really sure TBH. I just noticed this when doing my monthy system updates.

:computer: Components impacted

Dockerized ntfy server. Running behind Traefik, which in turn is behind Cloudflare proxy (it was a bitch to get to run properly at first, but it's been running fine for ages now).

:bulb: Screenshots and/or logs

❯ dps
    CONTAINER ID   NAMES            SIZE                     STATUS
    0123456789ab   container        0B (virtual 606MB)       Up 2 weeks
    83a65aa860c2   ntfy             0B (virtual 53.2MB)      Up 2 weeks (unhealthy)  
    0123456789ab   container        23.8kB (virtual 564MB)   Up 3 weeks
    629eb6192dc3   traefik          62.9kB (virtual 148MB)   Up 3 weeks
    0123456789ab   container        0B (virtual 6.27MB)      Exited (2) 4 weeks ago
    0123456789ab   container        78.8kB (virtual 765MB)   Up 3 weeks
    0123456789ab   container        64.9kB (virtual 765MB)   Up 3 weeks
    0123456789ab   container        2B (virtual 387MB)       Up 3 weeks
    0123456789ab   container        145kB (virtual 491MB)    Up 2 days
    0123456789ab   container        0B (virtual 38.7MB)      Up 3 weeks (healthy)
    0123456789ab   container        0B (virtual 412MB)       Up 3 weeks (healthy)
    0123456789ab   container        51.5MB (virtual 195MB)   Up 3 weeks
    0123456789ab   container        22.2kB (virtual 203MB)   Up 3 weeks

❯ pstree -p
systemd(1)-+-ModemManager(544846)-+
           ├─containerd-shim(1526704)─┬─ntfy(1526724)─┬─ssl_client(1527025)
           │                          │               ├─ssl_client(1527129)
           │                          │               ├─ssl_client(1527555)
           │                          │               ├─ssl_client(1527620)
           │                          │               ├─ssl_client(1527734)
           │                          │               ├─ssl_client(1527802)
           │                          │               ├─ssl_client([...])
           │                          │               ├─ssl_client(2700900)
           │                          │               ├─{ntfy}(1526760)
           │                          │               ├─{ntfy}(1526761)
           │                          │               ├─{ntfy}(1526762)
           │                          │               ├─{ntfy}(1526763)
           │                          │               ├─{ntfy}(1526764)
           │                          │               ├─{ntfy}(1526776)
           │                          │               ├─{ntfy}(1527782)
           │                          │               └─{ntfy}(1559488)

:crystal_ball: Additional context

Not really sure what I'm expecting with this post, perhaps that someone else has run into a similar thing or perhaps just to log my issue if it happens time and time again.

But basically, I have a few NTFY servers running here and there, this has happened on one of them and I don't think there are any config differences between them all.

After the initial headache getting them to run properly behind cloudflare's proxied DNS and traefik this was running fine. I then initiated an upgrade a few weeks ago (ie. 'docker compose pull && docker compose down ntfy && docker compose up ntfy -d') and made sure the service spun up again properly.

Have since left it alone as it's seemingly been working just fine - as in, my android client app is still receiving notifications just fine and has been doing so thruout. Only noticed this today when I was doing the monthly system updates. When I did notice the million or so defunct ssl sessions I immediately tried the web UI only to be greeted by a completely blank page, which makes no sense as the andorid client uses the web to connect, no? But either way, the web ui is no longer showing me data whereas the andoing app received a notification as recent as this morning.

I have since rebooted the entire server as there were kernel upgrades and the likes, but...

emigrating commented 9 months ago

Just logged for future.