AdGuard - CloudFlare - DoH sporadically Yes via 1.1.1.1/help

eltonajmenezes commented 2 years ago

Operating System

64-bit

Project

Cloudflare

Platform

Mac, IOS

Browser

Chrome, Other

Issue

Not working, Other (explain in description)

Issue Description

The first time you setup Cloudflare and point AdGuard to it; there seems to be no issue at all. Even the 1.1.1.1/help test always is consistent and indicated Yes for DoH and DoT.

But if a restart or shutdown happens then for some reason the status of clodflared when checked in terminal shows the following

flared[609]: 2022-04-30T10:48:21Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:18:21 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:48:21Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:18:21 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:48:21Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:18:21 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:48:21Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:18:21 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:48:21Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:18:21 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:48:21Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:18:21 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:48:21Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:18:22 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:48:22Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:18:22 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:48:22Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile> Apr 30 16:29:14 eltonsraspberrypi cloudflared[609]: 2022-04-30T10:59:14Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="faile>

These warnings can be 3 - 10 in number. If you stop the clodfalred service and restart it; the Warning goes away. But eventually it reappears.

Now the odd part is that around the same time this happens; the 1.1.1.1/help ,test gives you sporadically Yes for DoH when tested in Safari and Chrome browser compared to initially and either way the cache was cleared.

A9420696-468E-41C4-9374-99BB061C8902

https://user-images.githubusercontent.com/66936537/166317177-85b8b5f0-7461-4142-9c6c-0ea051c4a504.MOV

FA467290-7CD4-4121-81B3-B36D61B9CADD

I have searched multiple forums but no one’s solution seems to stick.

Also moving away from “Parallel Requests” to “Fastest IP Address” the issue goes but the speed of fetching web content is reduced by a small amount.

trinib commented 2 years ago

I'll have to test and see.. is it raspberry pi 64bit? you are using? cause 64bit came out February and could have issues.. if you do try 32bit and see if you get same errors

or it could be cloudflare version

eltonajmenezes commented 2 years ago

I'll have to test and see.. is it raspberry pi 64bit? you are using? cause 64bit came out February and could have issues.. if you do try 32bit and see if you get same errors

or it could be cloudflare version

Yes I was using 32Bit and this issue was present, moved to 64 Bit and the issue still happens

I am using the latest clodflared version present FYI

eltonajmenezes commented 2 years ago

I realized one more thing. Immediately after a restart the service seems to be fine and the behavior of 1.1.1.1/help DoH is also ok

eltonajmenezes commented 2 years ago

@eltonajmenezes I get no errors after reboot on Raspberry OS 64bit. and I saw no issues like yours on https://github.com/cloudflare/cloudflared

It's doesn't happen immediately. It happens all of a sudden, I will keep an eye and report back to you

trinib commented 2 years ago

@eltonajmenezes sorry I now saw the issues on https://github.com/cloudflare/cloudflared/issues/91 and https://github.com/cloudflare/cloudflared/issues/306.. I guess its something i cannot fix. I tried rebooting shutting down pi and rebooting router but issue do not shows for me still..

trinib commented 2 years ago

@eltonajmenezes use this method i suggested in https://github.com/trinib/AdGuard-WireGuard-Unbound-Cloudflare/issues/28#issuecomment-1115723760 for DoH. I guess I can add this to wiki as an alternative for Cloudflared tunnel client.

DNSCrypt - a DNS(DoH) proxy client.

eltonajmenezes commented 2 years ago

@eltonajmenezes use this method i suggested in https://github.com/trinib/AdGuard-WireGuard-Unbound-Cloudflare/issues/28#issuecomment-1115723760 for DoH. I guess I can add this to wiki as an alternative for Cloudflared tunnel client.

DNSCrypt - a DNS(DoH) proxy client.

I will try and replicate this and see what the dependencies are on my instance.

Maybe something is interfering. I will get back to you on this in a few days.

trinib commented 2 years ago

@eltonajmenezes I saw this person said they found a fix https://github.com/cloudflare/cloudflared/issues/306#issuecomment-989524710, try it and see if it works for you

open:

sudo nano /etc/default/cloudflared

add : --max-upstream-conns 50 zero

eltonajmenezes commented 2 years ago

@eltonajmenezes I saw this person said they found a fix https://github.com/cloudflare/cloudflared/issues/306#issuecomment-989524710, try it and see if it works for you

open:
sudo nano /etc/default/cloudflared
add : --max-upstream-conns 50

@trinib Yes I did try this in fact but it did not help at all.

I think I have narrowed down a possibility but need to observe and check.

trinib commented 2 years ago

it seems this issue happens for some and not everyone, some say it's a isp and router issue. https://github.com/cloudflare/cloudflared/issues/91#issuecomment-499912858

See if this works https://github.com/cloudflare/cloudflared/issues/91#issuecomment-491062486

eltonajmenezes commented 2 years ago

it seems this issue happens for some and not everyone, some say it's a isp and router issue. https://github.com/cloudflare/cloudflared/issues/91#issuecomment-499912858

See if this works https://github.com/cloudflare/cloudflared/issues/91#issuecomment-491062486

Doubt it's an ISP issue, and I do not have the hosts directory on my system.

trinib commented 2 years ago

it seems this issue happens for some and not everyone, some say it's a isp and router issue. cloudflare/cloudflared#91 (comment) See if this works cloudflare/cloudflared#91 (comment)

Doubt it's an ISP issue, and I do not have the hosts directory on my system.

you can create it .

trinib commented 2 years ago

This issue with cloudflare is really weird .. Hear this I have a good idea .. to really see if it's cloudfared or just your location isp router etc .. I want you to try on a VPS !!!. here is my personal referral link for free $35-https://www.vultr.com/?ref=9113990-8H for first user signup(limited). or if it do not work here is regular referral for $10-https://www.vultr.com/?ref=9113188

let me know if you need help setting it up .. it pretty easy and quick .. watch a youtube guide if issue

trinib commented 2 years ago

https://user-images.githubusercontent.com/18756975/166669805-4e5748ea-78fd-4906-8993-7df42607e920.mp4

eltonajmenezes commented 2 years ago

Thank you, will look into this and revert shortly. So you want me to test this on a virtual server right?

trinib commented 2 years ago

Thank you, will look into this and revert shortly. So you want me to test this on a virtual server right?

yes

eltonajmenezes commented 2 years ago

After using DNScrypt running on port 5335 as well I have started to see this issue where the upstream server sporadically works and stops most of the time.

Even after restarting the service I noticed this error wouldn't go away. May 12 18:11:37 eltonsraspberrypi dnscrypt-proxy[546859]: [2022-05-12 18:11:37] [ERROR] Get "https://dns.cloudflare.com/dns-query?dns=yv4BAAABAAAAAAABAAACAAEAACkQAAAAAAAAFAAMABDnWpigWoLlJKOn36NBJY-N": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

So I stopped the service, changed the port again to 53000 and started it.

but again it stopped I even did a test in this manner and I got a reply curl portquiz.net:5335 --connect-timeout 1

But when I tested this way dig sigfail.verteiltesysteme.net @127.0.0.1 -p 5335

I got a response ; <<>> DiG 9.16.27-Debian <<>> sigfail.verteiltesysteme.net @127.0.0.1 -p 5335 ;; global options: +cmd ;; connection timed out; no servers could be reached

trinib commented 2 years ago

After using DNScrypt running on port 5335 as well I have started to see this issue where the upstream server sporadically works and stops most of the time.

Even after restarting the service I noticed this error wouldn't go away. May 12 18:11:37 eltonsraspberrypi dnscrypt-proxy[546859]: [2022-05-12 18:11:37] [ERROR] Get "https://dns.cloudflare.com/dns-query?dns=yv4BAAABAAAAAAABAAACAAEAACkQAAAAAAAAFAAMABDnWpigWoLlJKOn36NBJY-N": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

So I stopped the service, changed the port again to 53000 and started it.

but again it stopped I even did a test in this manner and I got a reply curl portquiz.net:5335 --connect-timeout 1

But when I tested this way dig sigfail.verteiltesysteme.net @127.0.0.1 -p 5335

I got a response ; <<>> DiG 9.16.27-Debian <<>> sigfail.verteiltesysteme.net @127.0.0.1 -p 5335 ;; global options: +cmd ;; connection timed out; no servers could be reached

You need to show your configurations ..

trinib commented 2 years ago

for me i like to share my result unbound , cloudflare-proxy , dnscrypt-proxy working fine (form me i use family upstream you can use the properly you need)

2022-05-13_11-48-09 2022-05-13_11-48-35 2022-05-13_11-48-50

############################# Dig result as following 👍

 dig sigfail.verteiltesysteme.net @127.0.0.1 -p 5335 (Unbound dns)

; <<>> DiG 9.18.1-1ubuntu1-Ubuntu <<>> sigfail.verteiltesysteme.net @127.0.0.1 -p 5335
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 12068
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;sigfail.verteiltesysteme.net.  IN      A

;; Query time: 412 msec
;; SERVER: 127.0.0.1#5335(127.0.0.1) (UDP)
;; WHEN: Fri May 13 11:42:42 EEST 2022
;; MSG SIZE  rcvd: 57

 dig sigfail.verteiltesysteme.net @127.0.0.1 -p 5053

; <<>> DiG 9.18.1-1ubuntu1-Ubuntu <<>> sigfail.verteiltesysteme.net @127.0.0.1 -p 5053
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 56216
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: e48ef842b2bf56b5 (echoed)
;; QUESTION SECTION:
;sigfail.verteiltesysteme.net.  IN      A

;; Query time: 12 msec
;; SERVER: 127.0.0.1#5053(127.0.0.1) (UDP)
;; WHEN: Fri May 13 11:42:51 EEST 2022
;; MSG SIZE  rcvd: 69

 dig sigfail.verteiltesysteme.net @127.0.0.1 -p 6053  (dnscrypt-proxy)

; <<>> DiG 9.18.1-1ubuntu1-Ubuntu <<>> sigfail.verteiltesysteme.net @127.0.0.1 -p 6053
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 61243
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;sigfail.verteiltesysteme.net.  IN      A

;; Query time: 44 msec
;; SERVER: 127.0.0.1#6053(127.0.0.1) (UDP)
;; WHEN: Fri May 13 11:42:54 EEST 2022
;; MSG SIZE  rcvd: 57

@jo20201 what os and hardware are you using ? .. you get alot of errors for Unbound . It should look like this

.. I know the fix for the warning "warning: so-rcvbuf 1048576" . You need to open sudo nano /etc/sysctl.conf and add

net.core.rmem_max=1048576

I get that error on a VM Ubuntu, not sure about Pi at the moment(waiting on power supply).. those other errors are weird

trinib commented 2 years ago

for me i like to share my result unbound , cloudflare-proxy , dnscrypt-proxy working fine (form me i use family upstream you can use the properly you need) 2022-05-13_11-48-09 2022-05-13_11-48-35 2022-05-13_11-48-50 ############################# Dig result as following 👍

 dig sigfail.verteiltesysteme.net @127.0.0.1 -p 5335 (Unbound dns)

; <<>> DiG 9.18.1-1ubuntu1-Ubuntu <<>> sigfail.verteiltesysteme.net @127.0.0.1 -p 5335
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 12068
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;sigfail.verteiltesysteme.net.  IN      A

;; Query time: 412 msec
;; SERVER: 127.0.0.1#5335(127.0.0.1) (UDP)
;; WHEN: Fri May 13 11:42:42 EEST 2022
;; MSG SIZE  rcvd: 57

 dig sigfail.verteiltesysteme.net @127.0.0.1 -p 5053

; <<>> DiG 9.18.1-1ubuntu1-Ubuntu <<>> sigfail.verteiltesysteme.net @127.0.0.1 -p 5053
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 56216
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: e48ef842b2bf56b5 (echoed)
;; QUESTION SECTION:
;sigfail.verteiltesysteme.net.  IN      A

;; Query time: 12 msec
;; SERVER: 127.0.0.1#5053(127.0.0.1) (UDP)
;; WHEN: Fri May 13 11:42:51 EEST 2022
;; MSG SIZE  rcvd: 69

 dig sigfail.verteiltesysteme.net @127.0.0.1 -p 6053  (dnscrypt-proxy)

; <<>> DiG 9.18.1-1ubuntu1-Ubuntu <<>> sigfail.verteiltesysteme.net @127.0.0.1 -p 6053
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 61243
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;sigfail.verteiltesysteme.net.  IN      A

;; Query time: 44 msec
;; SERVER: 127.0.0.1#6053(127.0.0.1) (UDP)
;; WHEN: Fri May 13 11:42:54 EEST 2022
;; MSG SIZE  rcvd: 57

@jo20201 what os and hardware are you using ? .. you get alot of errors for Unbound . It should look like this .. I know the fix for the warning "warning: so-rcvbuf 1048576" . You need to open sudo nano /etc/sysctl.conf and add

net.core.rmem_max=1048576

I get that error on a VM Ubuntu, not sure about Pi at the moment(waiting on power supply).. those other errors are weird

Thx dude now the issue fixed with your help and I disable the log

What are you running Unbound on ? I still see these messages "notice: init module..."

trinib commented 2 years ago

this error in config file for unbound related with

 # Ensure kernel buffer is large enough to not lose messages in traffix spikes
    so-rcvbuf: 4m
    so-sndbuf: 4m

@jo20201 yea your right. 4m seems to work fine on PI. I do not know exactly why that happens in Ubuntu. you still have not said what are you running it on.

There is another way I figured out. You can set it in unbound.conf in kb. For example if it shows :

Open sudo nano /etc/unbound/unbound.conf.d/unbound.conf and set

Restart service:

sudo systemctl restart unbound

and no error

trinib commented 2 years ago

this error in config file for unbound related with
 # Ensure kernel buffer is large enough to not lose messages in traffix spikes
    so-rcvbuf: 4m
    so-sndbuf: 4m 
@jo20201 yea your right. 4m seems to work fine on PI. I do not know exactly why that happens in Ubuntu. you still have not said what are you running it on. There is another way I figured out. You can set it in unbound.conf in kb. For example if it shows : Open sudo nano /etc/unbound/unbound.conf.d/unbound.conf and set Restart service:
sudo systemctl restart unbound
and no error
my OS is ubuntu 22.04

@jo20201 what hardware ? no VM right ? if using PI, I guess its a Ubuntu thing ..

trinib commented 2 years ago

@eltonajmenezes i have been using it for a while and issue has not arrive for me.

trinib / AdGuard-WireGuard-Unbound-DNScrypt