search

cloudflare / cloudflared

Cloudflare Tunnel client (formerly Argo Tunnel)
https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide
Apache License 2.0
8.89k stars 781 forks source link

🐛 doesn't work when using Cloudflare WARP (VPN) #869

Closed dorianmariecom closed 1 year ago

dorianmariecom commented 1 year ago

Describe the bug

When running cloudflare warp and cloudflared, I can't reach the website being proxied by cloudflared

To Reproduce

  1. Configuration (.cloudflared.yml) 
    url: http://localhost:3000
tunnel: e9dde026-47f5-40eb-9a54-11db2f18e9a8
credentials-file: /Users/dorianmariefr/.cloudflared/e9dde026-47f5-40eb-9a54-11db2f18e9a8.json
  2. Run cloudflared tunnel --config .cloudflared.yml run
  3. See error: ERR Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=0 ip=198.41.200.13

If it's an issue with Cloudflare Tunnel:

  1. Tunnel ID : e9dde026-47f5-40eb-9a54-11db2f18e9a8
  2. cloudflared config: 
    url: http://localhost:3000
tunnel: e9dde026-47f5-40eb-9a54-11db2f18e9a8
credentials-file: /Users/dorianmariefr/.cloudflared/e9dde026-47f5-40eb-9a54-11db2f18e9a8.json

    Expected behavior

I can reach the website

Environment and versions

Logs and errors

23:30:58 web.1        | started with pid 70362
23:30:58 js.1         | started with pid 70363
23:30:58 css.1        | started with pid 70364
23:30:58 cloudflare.1 | started with pid 70365
23:30:58 js.1         | yarn run v1.22.19
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Starting tunnel tunnelID=e9dde026-47f5-40eb-9a54-11db2f18e9a8
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Version 2022.12.1
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF GOOS: darwin, GOVersion: go1.19.3, GoArch: amd64
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Settings: map[config:.cloudflared.yml cred-file:/Users/dorianmariefr/.cloudflared/e9dde026-47f5-40eb-9a54-11db2f18e9a8.json credentials-file:/Users/dorianmariefr/.cloudflared/e9dde026-47f5-40eb-9a54-11db2f18e9a8.json url:http://localhost:3000]
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Autoupdate frequency is set autoupdateFreq=86400000
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Generated Connector ID: e98031b3-2720-44b1-9da8-af1be5bacec6
23:30:58 js.1         | $ esbuild app/javascript/*.* --bundle --sourcemap --outdir=app/assets/builds --watch
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Initial protocol quic
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF ICMP proxy will use 192.168.1.13 as source for IPv4
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF ICMP proxy will use fe80::400:fad0:b3dc:ee8 in zone en0 as source for IPv6
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Created ICMP proxy listening on 192.168.1.13:0
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Created ICMP proxy listening on [fe80::400:fad0:b3dc:ee8%en0]:0
23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Starting metrics server on 127.0.0.1:50740/metrics
23:30:58 js.1         | [watch] build finished, watching for changes...
23:30:59 web.1        | => Booting Puma
23:30:59 web.1        | => Rails 7.0.4 application starting in development 
23:30:59 web.1        | => Run `bin/rails server --help` for more startup options
23:30:59 css.1        | ["/Users/dorianmariefr/.asdf/installs/ruby/3.1.3/lib/ruby/gems/3.1.0/gems/tailwindcss-rails-2.0.21-arm64-darwin/exe/arm64-darwin/tailwindcss", "-i", "/Users/dorianmariefr/src/pipelines/app/assets/stylesheets/application.tailwind.css", "-o", "/Users/dorianmariefr/src/pipelines/app/assets/builds/tailwind.css", "-c", "/Users/dorianmariefr/src/pipelines/config/tailwind.config.js", "--minify", "-w"]
23:30:59 web.1        | [70362] Puma starting in cluster mode...
23:30:59 web.1        | [70362] * Puma version: 6.0.2 (ruby 3.1.3-p185) ("Sunflower")
23:30:59 web.1        | [70362] *  Min threads: 5
23:30:59 web.1        | [70362] *  Max threads: 5
23:30:59 web.1        | [70362] *  Environment: development
23:30:59 web.1        | [70362] *   Master PID: 70362
23:30:59 web.1        | [70362] *      Workers: 2
23:30:59 web.1        | [70362] *     Restarts: (✔) hot (✖) phased
23:30:59 web.1        | [70362] * Preloading application
23:30:59 web.1        | [70362] * Listening on http://127.0.0.1:3000
23:30:59 web.1        | [70362] * Listening on http://[::1]:3000
23:30:59 web.1        | [70362] * Listening on http://127.0.2.2:3000
23:30:59 web.1        | [70362] * Listening on http://127.0.2.3:3000
23:30:59 web.1        | [70362] Use Ctrl-C to stop
23:30:59 web.1        | [70362] - Worker 0 (PID: 70369) booted in 0.0s, phase: 0
23:30:59 web.1        | [70362] - Worker 1 (PID: 70370) booted in 0.0s, phase: 0
23:31:00 css.1        | 
23:31:00 css.1        | Rebuilding...
23:31:00 css.1        | 
23:31:00 css.1        | Done in 332ms.
23:31:03 cloudflare.1 | 2023-01-09T22:31:03Z ERR Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=0 ip=198.41.200.13
23:31:03 cloudflare.1 | 2023-01-09T22:31:03Z INF Retrying connection in up to 2s connIndex=0 ip=198.41.200.13
23:31:08 cloudflare.1 | 2023-01-09T22:31:08Z ERR Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=0 ip=198.41.200.23
23:31:08 cloudflare.1 | 2023-01-09T22:31:08Z INF Retrying connection in up to 4s connIndex=0 ip=198.41.200.23
23:31:13 cloudflare.1 | 2023-01-09T22:31:13Z INF Connection bdbde909-7dc4-4eaa-b44b-61a5c8c81be5 registered with protocol: quic connIndex=0 ip=198.41.192.47 location=CDG
23:31:14 cloudflare.1 | 2023-01-09T22:31:14Z WRN Failed to serve quic connection error="already connected to this server, trying another address" connIndex=2 ip=198.41.192.167
23:31:14 cloudflare.1 | 2023-01-09T22:31:14Z WRN Unable to establish connection. error="already connected to this server, trying another address" connIndex=2 ip=198.41.192.167
23:31:14 cloudflare.1 | 2023-01-09T22:31:14Z INF Retrying connection in up to 2s connIndex=2 ip=198.41.192.167
23:31:15 cloudflare.1 | 2023-01-09T22:31:15Z WRN Failed to serve quic connection error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.57
23:31:15 cloudflare.1 | 2023-01-09T22:31:15Z WRN Unable to establish connection. error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.57
23:31:15 cloudflare.1 | 2023-01-09T22:31:15Z INF Retrying connection in up to 2s connIndex=3 ip=198.41.192.57
23:31:16 cloudflare.1 | 2023-01-09T22:31:16Z WRN Connection terminated error="already connected to this server, trying another address" connIndex=2
23:31:16 cloudflare.1 | 2023-01-09T22:31:16Z WRN Connection terminated error="already connected to this server, trying another address" connIndex=3
23:31:18 cloudflare.1 | 2023-01-09T22:31:18Z WRN Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=1 ip=198.41.200.63
23:31:18 cloudflare.1 | 2023-01-09T22:31:18Z INF Retrying connection in up to 2s connIndex=1 ip=198.41.200.63
23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z WRN Failed to serve quic connection error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.107
23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z WRN Unable to establish connection. error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.107
23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z INF Retrying connection in up to 4s connIndex=3 ip=198.41.192.107
23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z WRN Connection terminated error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=1
23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z WRN Connection terminated error="already connected to this server, trying another address" connIndex=3
23:31:21 cloudflare.1 | 2023-01-09T22:31:21Z WRN Failed to serve quic connection error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.227
23:31:21 cloudflare.1 | 2023-01-09T22:31:21Z WRN Unable to establish connection. error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.227
23:31:21 cloudflare.1 | 2023-01-09T22:31:21Z INF Retrying connection in up to 8s connIndex=3 ip=198.41.192.227
23:31:24 cloudflare.1 | 2023-01-09T22:31:24Z WRN Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=2 ip=198.41.200.233

Additional context Maybe this is a feature, but I would consider it more like a bug

cFiori415 commented 1 year ago

Please stay with your consequence - nan.

!NO-INSECTS.

On Mon, Jan 9, 2023, 5:37 PM Dorian Marié @.***> wrote:

Describe the bug

When running cloudflare warp and cloudflared, I can't reach the website being proxied by cloudflared

To Reproduce

  1. Configuration (.cloudflared.yml)

url: http://localhost:3000 tunnel: e9dde026-47f5-40eb-9a54-11db2f18e9a8 credentials-file: /Users/dorianmariefr/.cloudflared/e9dde026-47f5-40eb-9a54-11db2f18e9a8.json

  1. Run cloudflared tunnel --config .cloudflared.yml run
  2. See error: ERR Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=0 ip=198.41.200.13

If it's an issue with Cloudflare Tunnel:

  1. Tunnel ID : e9dde026-47f5-40eb-9a54-11db2f18e9a8
  2. cloudflared config:

url: http://localhost:3000 tunnel: e9dde026-47f5-40eb-9a54-11db2f18e9a8 credentials-file: /Users/dorianmariefr/.cloudflared/e9dde026-47f5-40eb-9a54-11db2f18e9a8.json

Expected behavior

I can reach the website

Environment and versions

  • OS: [e.g. MacOS] macOS Darwin computer.dorianmarie.fr 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:15:09 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T6000 arm64
  • Architecture: [e.g. AMD, ARM] ARM arm64
  • Version: [e.g. 2022.02.0] cloudflared version 2022.12.1 (built 2022-12-20-1251 UTC)

Logs and errors

23:30:58 web.1 | started with pid 70362

23:30:58 js.1 | started with pid 70363

23:30:58 css.1 | started with pid 70364

23:30:58 cloudflare.1 | started with pid 70365

23:30:58 js.1 | yarn run v1.22.19

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Starting tunnel tunnelID=e9dde026-47f5-40eb-9a54-11db2f18e9a8

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Version 2022.12.1

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF GOOS: darwin, GOVersion: go1.19.3, GoArch: amd64

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Settings: map[config:.cloudflared.yml cred-file:/Users/dorianmariefr/.cloudflared/e9dde026-47f5-40eb-9a54-11db2f18e9a8.json credentials-file:/Users/dorianmariefr/.cloudflared/e9dde026-47f5-40eb-9a54-11db2f18e9a8.json url:http://localhost:3000]

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Autoupdate frequency is set autoupdateFreq=86400000

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Generated Connector ID: e98031b3-2720-44b1-9da8-af1be5bacec6

23:30:58 js.1 | $ esbuild app/javascript/. --bundle --sourcemap --outdir=app/assets/builds --watch

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Initial protocol quic

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF ICMP proxy will use 192.168.1.13 as source for IPv4

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF ICMP proxy will use fe80::400:fad0:b3dc:ee8 in zone en0 as source for IPv6

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Created ICMP proxy listening on 192.168.1.13:0

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Created ICMP proxy listening on [fe80::400:fad0:b3dc:ee8%en0]:0

23:30:58 cloudflare.1 | 2023-01-09T22:30:58Z INF Starting metrics server on 127.0.0.1:50740/metrics

23:30:58 js.1 | [watch] build finished, watching for changes...

23:30:59 web.1 | => Booting Puma

23:30:59 web.1 | => Rails 7.0.4 application starting in development

23:30:59 web.1 | => Run bin/rails server --help for more startup options

23:30:59 css.1 | ["/Users/dorianmariefr/.asdf/installs/ruby/3.1.3/lib/ruby/gems/3.1.0/gems/tailwindcss-rails-2.0.21-arm64-darwin/exe/arm64-darwin/tailwindcss", "-i", "/Users/dorianmariefr/src/pipelines/app/assets/stylesheets/application.tailwind.css", "-o", "/Users/dorianmariefr/src/pipelines/app/assets/builds/tailwind.css", "-c", "/Users/dorianmariefr/src/pipelines/config/tailwind.config.js", "--minify", "-w"]

23:30:59 web.1 | [70362] Puma starting in cluster mode...

23:30:59 web.1 | [70362] * Puma version: 6.0.2 (ruby 3.1.3-p185) ("Sunflower")

23:30:59 web.1 | [70362] * Min threads: 5

23:30:59 web.1 | [70362] * Max threads: 5

23:30:59 web.1 | [70362] * Environment: development

23:30:59 web.1 | [70362] * Master PID: 70362

23:30:59 web.1 | [70362] * Workers: 2

23:30:59 web.1 | [70362] * Restarts: (✔) hot (✖) phased

23:30:59 web.1 | [70362] * Preloading application

23:30:59 web.1 | [70362] * Listening on http://127.0.0.1:3000

23:30:59 web.1 | [70362] * Listening on http://[::1]:3000

23:30:59 web.1 | [70362] * Listening on http://127.0.2.2:3000

23:30:59 web.1 | [70362] * Listening on http://127.0.2.3:3000

23:30:59 web.1 | [70362] Use Ctrl-C to stop

23:30:59 web.1 | [70362] - Worker 0 (PID: 70369) booted in 0.0s, phase: 0

23:30:59 web.1 | [70362] - Worker 1 (PID: 70370) booted in 0.0s, phase: 0

23:31:00 css.1 |

23:31:00 css.1 | Rebuilding...

23:31:00 css.1 |

23:31:00 css.1 | Done in 332ms.

23:31:03 cloudflare.1 | 2023-01-09T22:31:03Z ERR Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=0 ip=198.41.200.13

23:31:03 cloudflare.1 | 2023-01-09T22:31:03Z INF Retrying connection in up to 2s connIndex=0 ip=198.41.200.13

23:31:08 cloudflare.1 | 2023-01-09T22:31:08Z ERR Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=0 ip=198.41.200.23

23:31:08 cloudflare.1 | 2023-01-09T22:31:08Z INF Retrying connection in up to 4s connIndex=0 ip=198.41.200.23

23:31:13 cloudflare.1 | 2023-01-09T22:31:13Z INF Connection bdbde909-7dc4-4eaa-b44b-61a5c8c81be5 registered with protocol: quic connIndex=0 ip=198.41.192.47 location=CDG

23:31:14 cloudflare.1 | 2023-01-09T22:31:14Z WRN Failed to serve quic connection error="already connected to this server, trying another address" connIndex=2 ip=198.41.192.167

23:31:14 cloudflare.1 | 2023-01-09T22:31:14Z WRN Unable to establish connection. error="already connected to this server, trying another address" connIndex=2 ip=198.41.192.167

23:31:14 cloudflare.1 | 2023-01-09T22:31:14Z INF Retrying connection in up to 2s connIndex=2 ip=198.41.192.167

23:31:15 cloudflare.1 | 2023-01-09T22:31:15Z WRN Failed to serve quic connection error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.57

23:31:15 cloudflare.1 | 2023-01-09T22:31:15Z WRN Unable to establish connection. error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.57

23:31:15 cloudflare.1 | 2023-01-09T22:31:15Z INF Retrying connection in up to 2s connIndex=3 ip=198.41.192.57

23:31:16 cloudflare.1 | 2023-01-09T22:31:16Z WRN Connection terminated error="already connected to this server, trying another address" connIndex=2

23:31:16 cloudflare.1 | 2023-01-09T22:31:16Z WRN Connection terminated error="already connected to this server, trying another address" connIndex=3

23:31:18 cloudflare.1 | 2023-01-09T22:31:18Z WRN Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=1 ip=198.41.200.63

23:31:18 cloudflare.1 | 2023-01-09T22:31:18Z INF Retrying connection in up to 2s connIndex=1 ip=198.41.200.63

23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z WRN Failed to serve quic connection error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.107

23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z WRN Unable to establish connection. error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.107

23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z INF Retrying connection in up to 4s connIndex=3 ip=198.41.192.107

23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z WRN Connection terminated error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=1

23:31:19 cloudflare.1 | 2023-01-09T22:31:19Z WRN Connection terminated error="already connected to this server, trying another address" connIndex=3

23:31:21 cloudflare.1 | 2023-01-09T22:31:21Z WRN Failed to serve quic connection error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.227

23:31:21 cloudflare.1 | 2023-01-09T22:31:21Z WRN Unable to establish connection. error="already connected to this server, trying another address" connIndex=3 ip=198.41.192.227

23:31:21 cloudflare.1 | 2023-01-09T22:31:21Z INF Retrying connection in up to 8s connIndex=3 ip=198.41.192.227

23:31:24 cloudflare.1 | 2023-01-09T22:31:24Z WRN Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=2 ip=198.41.200.233

Additional context Maybe this is a feature, but I would consider it more like a bug

— Reply to this email directly, view it on GitHub https://github.com/cloudflare/cloudflared/issues/869, or unsubscribe https://github.com/notifications/unsubscribe-auth/A44U6XEIWS37VX3WZ6AUWL3WRSHJVANCNFSM6AAAAAATV6UZEU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

joliveirinha commented 1 year ago

Hi @dorianmariefr ,

This happens because of the architecture of both products. We were already aware of this and we are having an internal discussion to decide the way forward.

Note that, using Cloudflared and WARP is not ideal since in effect you would be in fact using a Tunnel over a Tunnel (WARP tunnel). Besides being inefficient, it has other limitations such as the fact that Cloudflared uses 4 connections to the edge for availability, but WARP uses 1, which defeats the high availability purpose of the 4 connections.

That said, Cloudflare Tunnel over WARP shouldn't be used for production tunnels. In the case you need to run WARP and Tunnel in the same machine, I suggest that you configure Split Tunnel configuration to exclude the Cloudflare Tunnel anycast IPs, which would circumvent this.

dorianmariecom commented 1 year ago

, I suggest that you configure Split Tunnel configuration to exclude the Cloudflare Tunnel anycast IPs, which would circumvent this.

How can I do that?

joliveirinha commented 1 year ago

You can take a look here.

https://developers.cloudflare.com/cloudflare-one/connections/connect-devices/warp/configure-warp/route-traffic/split-tunnels/#:~:text=Split%20Tunnels%20mode%20can%20be,the%20flow%20of%20IP%20traffic.

Regarding the IPs to exclude, take a look here. https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/do-more-with-tunnels/ports-and-ips/

benjamin-smith commented 8 months ago

Adding my use case for using WARP and CF Tunnels on the same machine.

Our engineering team has WARP installed on their Mac laptops, and need it to access internal company APIs and databases. We use WARP as a VPN in that sense.

When developing a web application on their devices, they will need to have their local web servers (nginx, apache, whatever) accessible over HTTP(S) behind Cloudflare Access, which we use extensively for authentication (via the Cf-Access-Jwt-Assertion headers/cookies).

Nothing that the workaround posted above does indeed allow us to use both WARP and Tunnels simultaneously. It was a bit of an adventure finding this GH issue however :)

dorianmariecom commented 8 months ago

To be honest I tried configuring WARP but couldn't even log in to my Cloudflare account with WARP. And I didn't figure out how to add the exceptions

dorianmariecom commented 5 months ago

Still an issue