Custom derp servers suddenly stopped working

bleomycin commented 3 months ago

What is the issue?

Tailscale status reports:
# Health check:
#     - not connected to home DERP region 900

I have been successfully running 2 custom derp servers on dedicated VPS's with no reverse proxy or firewall in their way for some time now. They pull certs directly from lets encrypt and currently have valid ssl certs issued. Suddenly after weeks of perfect operation my tailscale clients have all decided to throw the error for both of my derper servers: "not connected to home DERP region". I have changed nothing over this time except keep up with tailscale and derp software updates.

tailscale bugreports from multiple clients windows & linux:


BUG-7b5d81ccc107a2993dad6840fefdec1bbe97931a3c61080c3163277808984ba3-20240618010014Z-d0b7e7228fffa2d4
BUG-8e83084e5d3f5969935aa1e290975c2b21c677e1356ea78a6e079abcc221aebc-20240618010252Z-f675e4b5ea6961d8
BUG-1b447ddc63ecc78711e0577905ba0f7c59f6e2472189bda30ee64d9241cc5303-20240618010337Z-0e260b43db713052

ACL Snippet showing my configuration with domain names sanitized.

    // Custom DERP Relay
    "derpMap": {
        "OmitDefaultRegions": true,
        "Regions": {
            "900": {
                "RegionID":   900,
                "RegionCode": "relayla",
                "Nodes": [
                    {
                        "Name":     "1",
                        "RegionID": 900,
                        "HostName": "relayla.mydomain.com",
                    },
                ],
            },
            "901": {
                "RegionID":   901,
                "RegionCode": "relayaz",
                "Nodes": [
                    {
                        "Name":     "1",
                        "RegionID": 901,
                        "HostName": "relayaz.mydomain.com",
                    },
                ],
            },

Derper version for both custom servers:

go version go1.22.4 linux/amd64 
Derper Version: 1.69.0-ERR-BuildInfo

Steps to reproduce

No response

Are there any recent changes that introduced the issue?

No response

OS

Linux, macOS, Windows, iOS

OS version

windows 11, debian 12

Tailscale version

1.68.1

Other software

No response

Bug report

BUG-7b5d81ccc107a2993dad6840fefdec1bbe97931a3c61080c3163277808984ba3-20240618010014Z-d0b7e7228fffa2d4

bingoku commented 3 months ago

I also encountered this problem

alexl4321 commented 3 months ago

same. all tailnets with custom derps broke. nodes not reachable. when i connect to them through other networks they say "not connected to home derp xxx" in tailscale status.

bleomycin commented 3 months ago

Since the developers seem to have no interest in this does anyone else affected know if this has been resolved or should we be looking for alternatives?

cz19860416 commented 1 week ago

I also encounter this problem!!

tailscale / tailscale