search

aarond10 / https_dns_proxy

A lightweight DNS-over-HTTPS proxy.
MIT License
779 stars 114 forks source link

Doesn't work with >=curl-7.81.0 #136

Closed installgentoo closed 2 years ago

installgentoo commented 2 years ago

Errors with https_client.c:272 curl_multi_socket_action: 8 can't resolve anything, tested on google and cloudflare

installgentoo commented 2 years ago

curl built with flags:

 U I
 - - abi_x86_32       : 32-bit (x86) libraries
 - - adns             : Add support for asynchronous DNS resolution
 - - alt-svc          : Enable alt-svc support 
 - - brotli           : Enable brotli compression support 
 - - curl_ssl_gnutls  : Use GnuTLS
 - - curl_ssl_mbedtls : Use mbed TLS
 - - curl_ssl_nss     : Use Mozilla's Network Security Services
 + - curl_ssl_openssl : Use OpenSSL
 + - ftp              : Enable FTP support 
 - - gnutls           : Enable gnutls ssl backend 
 - - gopher           : Enable Gopher protocol support 
 - - hsts             : Enable HTTP Strict Transport Security 
 + - http2            : Enable HTTP/2.0 support 
 - - idn              : Enable support for Internationalized Domain Names
 + - imap             : Enable Internet Message Access Protocol support 
 - - ipv6             : Add support for IP version 6
 - - kerberos         : Add kerberos support
 - - ldap             : Add LDAP support (Lightweight Directory Access Protocol)
 - - mbedtls          : Enable mbedtls ssl backend 
 - - nss              : Enable nss ssl backend 
 - - openssl          : Enable openssl ssl backend 
 + - pop3             : Enable Post Office Protocol 3 support 
 + - progress-meter   : Enable the progress meter 
 - - quiche           : Enable HTTP/3.0 support using net-libs/quiche
 - - rtmp             : Enable RTMP Streaming Media support 
 - - samba            : Add support for SAMBA (Windows File and Printer sharing)
 + - smtp             : Enable Simple Mail Transfer Protocol support 
 - - ssh              : Enable SSH urls in curl using libssh2 
 + + ssl              : Enable crypto engine support (via openssl if USE='-gnutls -nss') 
 - - sslv3            : Support for the old/insecure SSLv3 protocol 
 - - static-libs      : Build static versions of dynamic libraries as well
 - - telnet           : Enable Telnet protocol support 
 - - test             : Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled independently)
 + - tftp             : Enable TFTP support 
 + + threads          : Add threads support for various packages. Usually pthreads
 + - zstd             : Enable zstd compression

threads probably the relevant one

ziyangcommon commented 2 years ago

I tested google,the same question

➜ https_dns_proxy git:(master) /usr/local/bin/https_dns_proxy -u nobody -g nogroup -v -v -r https://dns.google/dns-query -a 0.0.0.0 -p 53 -t socks5://192.168.1.1:8090 [I] 1650618987.917851 main.c:219 Version 2022.03.26-a63fea9 [I] 1650618987.917868 main.c:220 Built Apr 22 2022 09:04:46. [I] 1650618987.917870 main.c:221 System c-ares: 1.14.0 [I] 1650618987.917916 main.c:222 System libcurl: libcurl/7.64.0 OpenSSL/1.1.1n zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 librtmp/2.3 [W] 1650618987.918510 main.c:239 HTTP/3 was not available at build time, it will not work at all [I] 1650618987.918556 dns_server.c:50 Listening on 0.0.0.0:53 [I] 1650618987.918688 main.c:306 DNS polling initialized for 'dns.google' [W] 1650618998.006757 https_client.c:348 0001: curl request failed with 0: No error [W] 1650618998.006773 https_client.c:350 0001: curl error message: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dns.google:443 [W] 1650618998.006776 https_client.c:377 0001: No response (probably connection has been closed or timed out) [W] 1650618998.006780 https_client.c:413 0001: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [I] 1650618998.006782 https_client.c:501 0001: Response was faulty, skipping DNS reply. [W] 1650618999.937796 https_client.c:348 0002: curl request failed with 0: No error [W] 1650618999.937814 https_client.c:350 0002: curl error message: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dns.google:443 [W] 1650618999.937816 https_client.c:377 0002: No response (probably connection has been closed or timed out) [W] 1650618999.937819 https_client.c:413 0002: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [I] 1650618999.937821 https_client.c:501 0002: Response was faulty, skipping DNS reply. [W] 1650619001.946082 https_client.c:348 0003: curl request failed with 0: No error [W] 1650619001.946102 https_client.c:350 0003: curl error message: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dns.google:443 [W] 1650619001.946105 https_client.c:377 0003: No response (probably connection has been closed or timed out) [W] 1650619001.946109 https_client.c:413 0003: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [I] 1650619001.946111 https_client.c:501 0003: Response was faulty, skipping DNS reply. [W] 1650619003.959974 https_client.c:348 0004: curl request failed with 0: No error [W] 1650619003.959991 https_client.c:350 0004: curl error message: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dns.google:443 [W] 1650619003.959994 https_client.c:377 0004: No response (probably connection has been closed or timed out) [W] 1650619003.959997 https_client.c:413 0004: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [I] 1650619003.959999 https_client.c:501 0004: Response was faulty, skipping DNS reply. [W] 1650619005.971672 https_client.c:348 0005: curl request failed with 0: No error [W] 1650619005.971690 https_client.c:350 0005: curl error message: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dns.google:443 [W] 1650619005.971694 https_client.c:377 0005: No response (probably connection has been closed or timed out) [W] 1650619005.971695 https_client.c:413 0005: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [I] 1650619005.971707 https_client.c:501 0005: Response was faulty, skipping DNS reply. [W] 1650619105.639249 https_client.c:348 0001: curl request failed with 0: No error

baranyaib90 commented 2 years ago

Errors with https_client.c:272 curl_multi_socket_action: 8 can't resolve anything, tested on google and cloudflare

I have build curl 7.83.0 with HTTP 2 and 3 support and the proxy works fine with it. I don't have any idea how did you build it and why it's not working.

baranyaib90 commented 2 years ago

ziyangcommon According to your logs:

You could first test simple curl command, like: curl -v -m 10 -x socks5://192.168.1.1:8090 -http2 https://dns.google/dns-query

installgentoo commented 2 years ago

@baranyaib90 I rebuilt with hsts, quiche for http3, openssl for ssl. same error. what changed in 7.81.0 ?

ziyangcommon commented 2 years ago

ziyangcommon According to your logs:

  • libcurl/7.64.0 is used, which is fully supported
  • It seems like, in your case "No response (probably connection has been closed or timed out)" is the main problem. This means, that the request was sent, but no response was returned. This might be connection issue, check proxy or connection.

You could first test simple curl command, like: curl -v -m 10 -x socks5://192.168.1.1:8090 -http2 https://dns.google/dns-query

I tested with google it reachable,but https_dns_proxy still won't work

➜ https_dns_proxy git:(master) curl -v -x socks5://192.168.1.1:8090 --http2 https://dns.google/dns-query

➜ https_dns_proxy git:(master) /usr/local/bin/https_dns_proxy -u nobody -g nogroup -v -v -r https://dns.google/dns-query -a 0.0.0.0 -p 5353 -t socks5://192.168.1.1:8090 [I] 1655449358.793316 main.c:219 Version 2022.03.26-a63fea9 [I] 1655449358.793329 main.c:220 Built Apr 22 2022 09:04:46. [I] 1655449358.793331 main.c:221 System c-ares: 1.14.0 [I] 1655449358.793368 main.c:222 System libcurl: libcurl/7.64.0 OpenSSL/1.1.1n zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 librtmp/2.3 [W] 1655449358.793953 main.c:239 HTTP/3 was not available at build time, it will not work at all [I] 1655449358.793994 dns_server.c:50 Listening on 0.0.0.0:5353 [I] 1655449358.794117 main.c:306 DNS polling initialized for 'dns.google' [W] 1655449363.825273 https_client.c:348 C14E: curl request failed with 0: No error [W] 1655449363.825290 https_client.c:350 C14E: curl error message: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dns.google:443 [W] 1655449363.825294 https_client.c:377 C14E: No response (probably connection has been closed or timed out) [W] 1655449363.825298 https_client.c:413 C14E: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [I] 1655449363.825302 https_client.c:501 C14E: Response was faulty, skipping DNS reply. [W] 1655449368.756301 https_client.c:348 C14E: curl request failed with 0: No error [W] 1655449368.756318 https_client.c:350 C14E: curl error message: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dns.google:443 [W] 1655449368.756321 https_client.c:377 C14E: No response (probably connection has been closed or timed out) [W] 1655449368.756324 https_client.c:413 C14E: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [I] 1655449368.756326 https_client.c:501 C14E: Response was faulty, skipping DNS reply. [W] 1655449373.770171 https_client.c:348 C14E: curl request failed with 0: No error [W] 1655449373.770186 https_client.c:350 C14E: curl error message: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to dns.google:443 [W] 1655449373.770188 https_client.c:377 C14E: No response (probably connection has been closed or timed out) [W] 1655449373.770191 https_client.c:413 C14E: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [I] 1655449373.770193 https_client.c:501 C14E: Response was faulty, skipping DNS reply.

baranyaib90 commented 2 years ago

Hi @ziyangcommon , first use -v -v -v to see debug logs. Most likely the problem is, that you also have to add -4 option to use IPv4.

aarond10 commented 2 years ago

Please reopen if this is still an issue.

WRW001 commented 2 years ago

Please reopen if this is still an issue.

Hello! This issue occurred today when I had my server updated to Fedora 37. The detailed curl version info: curl --version curl 7.85.0 (x86_64-redhat-linux-gnu) libcurl/7.85.0 OpenSSL/3.0.5 zlib/1.2.12 brotli/1.0.9 libidn2/2.3.3 libpsl/0.21.1 (+libidn2/2.3.3) libssh/0.10.4/openssl/zlib nghttp2/1.47.0 Release-Date: 2022-08-31 Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets

The DoH_Proxy works fine on Fedora 36(curl_version:7.82.0)

Log info: ./https_dns_proxy -u nobody -g nobody -v -v -v -v -r https://ordns.he.net/dns-query -a [ip-listen] -p 53 -t socks5h://[proxy_ip:proxy_port] -4
[I] 1665757884.252844 main.c:219 Version 2022.08.13-a344d19 [I] 1665757884.252855 main.c:220 Built Oct 14 2022 22:27:44. [I] 1665757884.252856 main.c:221 System c-ares: 1.17.2 [I] 1665757884.252891 main.c:222 System libcurl: libcurl/7.85.0 OpenSSL/3.0.5 zlib/1.2.12 brotli/1.0.9 libidn2/2.3.3 libpsl/0.21.1 (+libidn2/2.3.3) libssh/0.10.4/openssl/zlib nghttp2/1.47.0 [W] 1665757884.253434 main.c:236 HTTP/3 is not supported by current libcurl

[D] 1665757884.253506 logging.c:39 starting periodic log flush timer [D] 1665757888.257014 main.c:113 Received request for id: 104, len: 26 [D] 1665757888.257076 https_client.c:253 0104: Requesting HTTP/2 [D] 1665757888.257089 https_client.c:322 0104: Using curl proxy: socks5h://[proxy_ip:proxy_port] [D] 1665757888.257252 https_client.c:79 curl opened socket: 8 [D] 1665757888.257274 https_client.c:207 0104: Trying [proxy_ip:proxy_port]... [D] 1665757888.257592 https_client.c:599 Reserved new io event: 0x7ffdff9e9f80 [D] 1665757888.257723 https_client.c:207 0104: SOCKS5 connect to ordns.he.net:443 (remotely resolved) [D] 1665757888.257808 https_client.c:207 0104: SOCKS5 request granted. [D] 1665757888.257827 https_client.c:207 0104: Connected to proxy_ip (proxy_ip) port proxy_port (#0) [D] 1665757888.262595 https_client.c:207 0104: ALPN: offers h2 [D] 1665757888.262615 https_client.c:207 0104: ALPN: offers http/1.1 [D] 1665757888.262643 https_client.c:207 0104: * error setting certificate file: }

[D] 1665757888.262661 https_client.c:207 0104: * Closing connection 0 [D] 1665757888.262697 https_client.c:589 Released used io event: 0x7ffdff9e9f80 [D] 1665757888.262773 https_client.c:114 curl closed socket: 8 [W] 1665757888.262795 https_client.c:353 0104: curl request failed with 0: No error [W] 1665757888.262802 https_client.c:355 0104: curl error message: error setting certificate file: }

[W] 1665757888.262810 https_client.c:382 0104: No response (probably connection has been closed or timed out) [W] 1665757888.262817 https_client.c:418 0104: CURLINFO_SSL_VERIFYRESULT: Unsupported protocol [D] 1665757888.262821 https_client.c:437 0104: CURLINFO_NUM_CONNECTS: 1 [D] 1665757888.262828 https_client.c:449 0104: CURLINFO_EFFECTIVE_URL: https://ordns.he.net/dns-query [D] 1665757888.262834 https_client.c:484 0104: Times: 0.000080, 0.000673, 0.000000, 0.000000, 0.000000, 0.005536 [I] 1665757888.262853 https_client.c:506 0104: Response was faulty, skipping DNS reply. [D] 1665757888.262859 main.c:84 Received response for id: 104, len: 0

WRW001 commented 2 years ago

ziyangcommon According to your logs:

  • libcurl/7.64.0 is used, which is fully supported
  • It seems like, in your case "No response (probably connection has been closed or timed out)" is the main problem. This means, that the request was sent, but no response was returned. This might be connection issue, check proxy or connection.

You could first test simple curl command, like: curl -v -m 10 -x socks5://192.168.1.1:8090 -http2 https://dns.google/dns-query

1.With "-http2", abnormal: curl -v -m 10 -x socks5://proxy_ip:proxy_port -http2 https://dns.google/dns-query Usage: curl [options...] Invalid category provided, here is a list of all categories:

auth Different types of authentication methods connection Low level networking operations curl The command line tool itself dns General DNS options file FILE protocol options ftp FTP protocol options http HTTP and HTTPS protocol options imap IMAP protocol options misc Options that don't fit into any other category output Filesystem output pop3 POP3 protocol options post HTTP Post specific options proxy All options related to proxies scp SCP protocol options sftp SFTP protocol options smtp SMTP protocol options ssh SSH protocol options telnet TELNET protocol options tftp TFTP protocol options tls All TLS/SSL related options upload All options for uploads verbose Options related to any kind of command line output of curl

  1. With "--http2", OK: curl -v -m 10 -x socks5://proxy_ip:proxy_port --http2 https://dns.google/dns-query
    • Trying proxy_ip:proxy_port...
    • SOCKS5 connect to IPv4 8.8.8.8:443 (locally resolved)
    • SOCKS5 request granted.
    • Connected to proxy_ip (proxy_ip) port proxy_port (#0)
    • ALPN: offers h2
    • ALPN: offers http/1.1
    • CAfile: /etc/pki/tls/certs/ca-bundle.crt
    • CApath: none
    • TLSv1.0 (OUT), TLS header, Certificate Status (22):
    • TLSv1.3 (OUT), TLS handshake, Client hello (1):
    • TLSv1.2 (IN), TLS header, Certificate Status (22):
    • TLSv1.3 (IN), TLS handshake, Server hello (2):
    • TLSv1.2 (IN), TLS header, Finished (20):
    • TLSv1.2 (IN), TLS header, Supplemental data (23):
    • TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
    • TLSv1.3 (IN), TLS handshake, Certificate (11):
    • TLSv1.3 (IN), TLS handshake, CERT verify (15):
    • TLSv1.3 (IN), TLS handshake, Finished (20):
    • TLSv1.2 (OUT), TLS header, Finished (20):
    • TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
    • TLSv1.2 (OUT), TLS header, Supplemental data (23):
    • TLSv1.3 (OUT), TLS handshake, Finished (20):
    • SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
    • ALPN: server accepted h2
    • Server certificate:
    • subject: CN=dns.google
    • start date: Sep 12 08:19:35 2022 GMT
    • expire date: Dec 5 08:19:34 2022 GMT
    • subjectAltName: host "dns.google" matched cert's "dns.google"
    • issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1C3
    • SSL certificate verify ok.
    • Using HTTP2, server supports multiplexing
    • Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
    • TLSv1.2 (OUT), TLS header, Supplemental data (23):
    • TLSv1.2 (OUT), TLS header, Supplemental data (23):
    • TLSv1.2 (OUT), TLS header, Supplemental data (23):
    • h2h3 [:method: GET]
    • h2h3 [:path: /dns-query]
    • h2h3 [:scheme: https]
    • h2h3 [:authority: dns.google]
    • h2h3 [user-agent: curl/7.85.0]
    • h2h3 [accept: /]
    • Using Stream ID: 1 (easy handle 0x55d20ebc1c20)
    • TLSv1.2 (OUT), TLS header, Supplemental data (23):

      GET /dns-query HTTP/2 Host: dns.google user-agent: curl/7.85.0 accept: /

    • TLSv1.2 (IN), TLS header, Supplemental data (23):
    • TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
    • TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
    • old SSL session ID is stale, removing
    • TLSv1.2 (IN), TLS header, Supplemental data (23):
    • TLSv1.2 (OUT), TLS header, Supplemental data (23):
    • TLSv1.2 (IN), TLS header, Supplemental data (23):
    • TLSv1.2 (IN), TLS header, Supplemental data (23): < HTTP/2 400 < x-content-type-options: nosniff < strict-transport-security: max-age=31536000; includeSubDomains; preload < access-control-allow-origin: * < date: Sat, 15 Oct 2022 03:57:09 GMT < content-type: text/html; charset=UTF-8 < server: HTTP server (unknown) < content-length: 1600 < x-xss-protection: 0 < x-frame-options: SAMEORIGIN < alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43" <
    • TLSv1.2 (IN), TLS header, Supplemental data (23): <!DOCTYPE html> Error 400 (Bad Request)!!1
    • TLSv1.2 (IN), TLS header, Supplemental data (23):

      400. That’s an error.

      Your client has issued a malformed or illegal request. Query must have a valid ‘dns’ parameter.. That’s all we know.

    • TLSv1.2 (OUT), TLS header, Supplemental data (23):
    • Connection #0 to host 10.20.30.99 left intact
baranyaib90 commented 2 years ago

Hi, because of "curl error message: error setting certificate file: }" please try first to fallback to commit https://github.com/aarond10/https_dns_proxy/commit/d310a378795790350703673388821558163944de

WRW001 commented 2 years ago

d310a37

Hi, baranyaib90 Everything works fine after I had the code fall back to this commit and recompiled the binary.

And a new issue will be filed since this is not a curl version issue.

aarond10 commented 2 years ago

ca_info is not initialized to nullptr if '-C' is not set. Let me fix that now.

On Sat, 15 Oct 2022 at 20:43, WRW001 @.***> wrote:

d310a37 https://github.com/aarond10/https_dns_proxy/commit/d310a378795790350703673388821558163944de

Hi, baranyaib90 https://github.com/baranyaib90 Everything works fine after I had the code fall back to this commit and recompiled the binary.

— Reply to this email directly, view it on GitHub https://github.com/aarond10/https_dns_proxy/issues/136#issuecomment-1279707299, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABTOXUPS5WYJPQ5SEWBCRDWDJ4FRANCNFSM5Q5T25EQ . You are receiving this because you modified the open/close state.Message ID: @.***>

aarond10 commented 2 years ago

Please try to sync and build now.

On Sat, 15 Oct 2022 at 20:55, Aaron Drew @.***> wrote:

ca_info is not initialized to nullptr if '-C' is not set. Let me fix that now.

On Sat, 15 Oct 2022 at 20:43, WRW001 @.***> wrote:

d310a37 https://github.com/aarond10/https_dns_proxy/commit/d310a378795790350703673388821558163944de

Hi, baranyaib90 https://github.com/baranyaib90 Everything works fine after I had the code fall back to this commit and recompiled the binary.

— Reply to this email directly, view it on GitHub https://github.com/aarond10/https_dns_proxy/issues/136#issuecomment-1279707299, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABTOXUPS5WYJPQ5SEWBCRDWDJ4FRANCNFSM5Q5T25EQ . You are receiving this because you modified the open/close state.Message ID: @.***>

WRW001 commented 2 years ago

Please try to sync and build now. … On Sat, 15 Oct 2022 at 20:55, Aaron Drew @.> wrote: ca_info is not initialized to nullptr if '-C' is not set. Let me fix that now. On Sat, 15 Oct 2022 at 20:43, WRW001 @.> wrote: > d310a37 > <d310a37> > > Hi, baranyaib90 https://github.com/baranyaib90 > Everything works fine after I had the code fall back to this commit and > recompiled the binary. > > — > Reply to this email directly, view it on GitHub > <#136 (comment)>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/AABTOXUPS5WYJPQ5SEWBCRDWDJ4FRANCNFSM5Q5T25EQ > . > You are receiving this because you modified the open/close state.Message > ID: @.***> >

Thank you for your lighting-fast fix aarond10!

Everything is OK, now. Commit f52a85f did the trick. This issue has been resolved!