[Bug] "Could not get the create map update"

crimewaffle commented 3 weeks ago

Is this a support request?

[X] This is not a support request

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

When trying to register a node I get the following error from headscale: ERR ../../../home/runner/work/headscale/headscale/hscontrol/poll.go:719 > Could not get the create map update error="invalid action" node=net.example.com node.id=1 omitPeers=false readOnly=false stream=true

The command I used: tailscale up --login-server https://net.example.com --advertise-exit-node=true --hostname=net.example.com --accept-dns=true

However headscale node ls outputs:

ID | Hostname     | Name         | MachineKey | NodeKey | User  | IP addresses | Ephemeral | Last seen           | Expiration          | Connected | Expired
1  | net.example.com | net.example.com | [n04ss]    | [AWCgQ] | example | 100.64.0.1,  | false     | 2024-08-22 18:18:24 | 0001-01-01 00:00:00 | offline   | no

Expected Behavior

The node should register and connect to the tailnet (or headnet?) correctly

Steps To Reproduce

Unsure. Just try to register a node.

Environment

- OS: Debian 12
- Headscale version: v0.23.0-beta2
- Tailscale version: 1.72.1

Runtime environment

[X] Headscale is behind a (reverse) proxy
[ ] Headscale runs in a container

Anything else?

I'm using caddy (version: 2.6.2) as a reverse proxy

stblassitude commented 3 weeks ago

I'm gettting these errors in the log for (some) of the existing clients:

headscale-1  | 2024-08-23T14:59:57Z INF home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has connected, mapSession: 0xc000731380, chan: 0xc0004ba2a0 node=bluebox node.id=6 omitPeers=false readOnly=false stream=true
headscale-1  | 2024-08-23T14:59:57Z ERR home/runner/work/headscale/headscale/hscontrol/poll.go:719 > Could not get the create map update error="invalid action" node=bluebox node.id=6 omitPeers=false readOnly=false stream=true
headscale-1  | 2024-08-23T14:59:57Z INF home/runner/work/headscale/headscale/hscontrol/poll.go:699 > node has disconnected, mapSession: 0xc000731380, chan: 0xc0004ba2a0 node=bluebox node.id=6 omitPeers=false readOnly=false stream=true

stblassitude commented 3 weeks ago

It turns out that I made a change to the acls just before upgrading, and that causes the Could not get the create map update error="invalid action" message. So the error handling for ACLs probably could use some improvements.

stblassitude commented 3 weeks ago

Specifcally, I spelled "acceept" incorrectly. It would be great if the ACL loader would do a sanity check on the contents.

crimewaffle commented 3 weeks ago

I think I had a similar problem. I had deny as an action and haven't restarted headscale after changing the ACLs. The upgrade caused the service to restart and fail.

In addition to what @stblassitude mentioned, I think it would be great if the server would just ignore the ACL file, and notify the admin if the sanity check fails. This would prevent that users/servers loose access to the tailnet.

kradalby commented 3 weeks ago

Is this issue present in beta1 and some of the latest alphas?

I'm gonna tag a new beta with some other fixes but won't be able to improve this until next week.

stblassitude commented 3 weeks ago

It's definitely in -beta2, I'm not sure if it was in -beta1.

kradalby commented 3 weeks ago

Would be great if I could have a couple of minimal reproducible example acls and if you can test some other version!

stblassitude commented 3 weeks ago

Both of the following ACLs trigger errors.

Could not get the create map update error="invalid action":

{
"acls": [
{ "action": "acceept", "src": ["testb"], "dst": ["testb:*"]},
]
}

# headscale policy set -f /etc/headscale/broken-acceept.json
2024-08-23T18:00:22Z WRN 
WARN: The "dns.use_username_in_magic_dns" configuration key is deprecated and has been removed. Please see the changelog for more details.

Could not get the create map update error="failed to parse destination, tokens [testb]: invalid port format":

{
"acls": [
{ "action": "accept", "src": ["testb"], "dst": ["testb"]},
]
}


# headscale policy set -f /etc/headscale/broken-missing-port.json
2024-08-23T18:02:31Z WRN 
WARN: The "dns.use_username_in_magic_dns" configuration key is deprecated and has been removed. Please see the changelog for more details.

Policy updated.



Also, I don't have `dns.use_username_in_magic_dns` in my config anymore, so the warning is confusing.

stblassitude commented 3 weeks ago

I think the main improvement would be to have an error message that mentions ACLs. And I'm happy to help test a newer version. I'm running the docker image.

juanfont / headscale