cloudflare / cloudflared

Cloudflare Tunnel client (formerly Argo Tunnel)
https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide
Apache License 2.0
9.22k stars 814 forks source link

💡 icmp over quic support #726

Closed 33Fraise33 closed 1 year ago

33Fraise33 commented 2 years ago

Describe the feature you'd like We currently run cloudflared quite distributed through our network but one of the main missing factors is the option to send icmp echo packets towards a device to check availability.

Describe alternatives you've considered We currently have a linux host we put available through cloudflared. That host is running from the same range as cloudflared is so there we can ping the devices as we have the same firewall rights configured. This is quite a dirty workaround and doesn't scale nice.

Additional context

sudarshan-reddy commented 2 years ago

This is something we are actually working on. I'll let @abelinkinbio track this.

abelinkinbio commented 2 years ago

Thanks for raising this as a feature request @33Fraise33. As @sudarshan-reddy mentioned, we're actively working on adding support for ICMP echo request and replies for Tunnel private network deployments. We'll have more to share on this in a few weeks (late Sept/ early Oct). Stay tuned 🙂

33Fraise33 commented 2 years ago

Thanks for raising this as a feature request @33Fraise33. As @sudarshan-reddy mentioned, we're actively working on adding support for ICMP echo request and replies for Tunnel private network deployments. We'll have more to share on this in a few weeks (late Sept/ early Oct). Stay tuned slightly_smiling_face

Thanks for the reply! I will further follow up with our customer manager.

racksync commented 2 years ago

This is something we are actually working on. I'll let @abelinkinbio track this.

Awesome!

33Fraise33 commented 2 years ago

Hi @abelinkinbio do you have any update on this, I notice that 2022.9.0 and 2022.9.1 mention icmp proxying but for the moment that is not working for us. Is there a specific item we need to configure or similar?

bjornfro commented 2 years ago

I noticed in the cloudflared log that I get this on Rocky Linux 8.

{"level":"warn","error":"cannot create ICMPv4 proxy: socket: permission denied nor ICMPv6 proxy: socket: permission denied","time":"2022-09-26T09:25:14Z","message":"Failed to create icmp router, ICMP proxy feature is disabled"}

Supposed to work? Which socket permission?

nmldiegues commented 2 years ago

I notice that 2022.9.0 and 2022.9.1 mention icmp proxying but for the moment that is not working for us. Is there a specific item we need to configure or similar?

This is expected for now, since we haven't released/announced the feature, so it is gated on our side.

I noticed in the cloudflared log that I get this on Rocky Linux 8.

This is because in Linux it requires the user, which owns the cloudflared process, to have ping_group_range capability (see https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt or https://github.com/ValentinBELYN/icmplib/blob/main/docs/6-use-icmplib-without-privileges.md )

This will be properly documented once the feature is released and usable. As noted above, you won't be able to get ICMP proxied anyway even if you address this for now.

bjornfro commented 2 years ago

On this machine cloudflared runs as root. You mean that is not enough?

nmldiegues commented 2 years ago

As per my link above:

ping_group_range - 2 INTEGERS Restrict ICMP_PROTO datagram sockets to users in the group range. The default is "1 0"

So root should be enough (since it runs with GID 0). But yet, the warning you should says "socket: permission denied"

bjornfro commented 1 year ago

What is actually required to make ICMP(ping) work? Not very important but handy to check availability and latency.

We got it enabled in our account. Running latest WARP and cloudflared. But, still no go.

Not sure if ICMP traceroute is reliable but it seems as if once traffic reaches Cloudflare, it is sent to public Internet, not to Cloudflare tunnel(cloudflared):

traceroute -I -n "internal IP" traceroute to ....), 64 hops max, 72 byte packets 1 172.16.0.1 8.331 ms 10.100 ms 5.981 ms 2 * 162.158.180.1 7.594 ms 6.081 ms 3 62.115.46.92 7.590 ms 5.583 ms 7.910 ms (ISP network) 4 62.115.125.248 7.464 ms 8.050 ms 6.157 ms 5 62.115.123.114 9.025 ms 33.299 ms 8.374 ms

33Fraise33 commented 1 year ago

What is needed to enable the icmp feature? I do not see it under network options in our account, only TCP and UDP tunneling are there.

abelinkinbio commented 1 year ago

@bjornfro I believe we have a ticket opened internally and are tracking your issue separately. However, let me know if that is not the case!

@33Fraise33 this feature is currently in closed beta for testing, but if you shoot me a quick email with your account_tag then I'd be happy to turn this on for you as well 🙂

bjornfro commented 1 year ago

It works, but limited to RFC 1918 destinations for now which was my issue. Might be good for others to know.

BillyILT commented 1 year ago

we are testing this as a alternative to our current VPN and came across this issue with ping as well and is there any news on this feature being enabled yet

bjornfro commented 1 year ago

It is enabled but only works with RFC 1918 ip-addresses AFAIK.

BillyILT commented 1 year ago

ok maybe we have something set up wrong as we have tunnels to our sites with routes and we can access internal resources but cant ping any devices which we know will respond.

bjornfro commented 1 year ago

cloudflared log should show if ICMP proxy was setup OK, plus is what you're pinging RFC 1918 dest IP?

abelinkinbio commented 1 year ago

@bjornfro that's interesting. Thanks for flagging. ICMP should be supported for both public and private endpoints. If you're unable to ping a public endpoint from a WARP enabled device please do file a support ticket and we'll be sure to take a look.

@BillyILT with other users, we've noted that permissions on the box running cloudflared get in the way of enabling ICMP. Here is a guide that walks through the groups/permissions required which may help.

cc: @obezuk

bjornfro commented 1 year ago

When was support for pinging "public"(non RFC 1918) IP via warp/cloudflared added? I was told it was to come later. Still don't work for me.

BillyILT commented 1 year ago

hi i followed the guide you sent and have rebooted the server as well but still no luck. Just in case i got this wrong we have a server acting as the tunnel endpoint in our network and our warp clients can use this to get to our internal private subnets. now this works fine i can https, ssh and RDP for example to other endpoints but i cant ping any from my warp client to any internal resources any other thoughts why this wont work result of the sysctl net.ipv4.ping_group_range command does return the what we expect

abelinkinbio commented 1 year ago

@bjornfro would you mind opening a support ticket? That's odd that you still aren't able to ping public destination with WARP enabled.

@BillyILT would you mind sharing the output of sysctl net.ipv4.ping_group_range?

bjornfro commented 1 year ago

I've had a ticket on this and then the response I got was that public IP's are not supported. I think I got Q2 or something as response when it might be supported. You can see a difference with icmp traceroute where the packet takes a different path , to public Internet when ip is public. Traffic is not sent to cloudflared tunnel. Maybe something with the WARP client.

traceroute -I -n 10.223.255.255 traceroute to 10.223.255.255 (10.223.255.255), 64 hops max, 72 byte packets 1 162.158.180.176 24.550 ms 10.670 ms 12.198 ms 2 172.71.97.86 215.374 ms 34.036 ms 32.526 ms 3 xxxx 4 10.223.255.255 56.866 ms 57.648 ms 57.190 ms

traceroute -I -n traceroute to 153.88.139.1 (153.88.139.1), 64 hops max, 72 byte packets 1 172.16.0.1 22.166 ms 13.525 ms 13.938 ms 2 162.158.180.1 18.272 ms 20.102 ms 10.926 ms 3 62.115.46.92 12.279 ms 11.581 ms 10.130 ms 4 62.115.125.248 17.316 ms 13.975 ms 10.409 ms 5 62.115.136.37 12.434 ms 13.417 ms 13.546 ms 6 62.115.155.53 10.964 ms 13.234 ms 10.135 ms .....fail

BillyILT commented 1 year ago

sysctl net.ipv4.ping_group_range net.ipv4.ping_group_range = 0 2147483647