search

bol-van / zapret

DPI bypass multi platform
5.95k stars 514 forks source link

Не работает files.catbox.moe с GnuTLS с TLS 1.3 #404

Open datissi opened 3 days ago

datissi commented 3 days ago

При использовании библиотеки GnuTLS вместо OpenSSL и версии TLS 1.3 происходит разрыв соедиения с файлообмеником files.catbox.moe.

Zapret запущен непосредственно на компьютере. Конфиг zapret следующий:

# this file is included from init scripts
# change values here

# can help in case /tmp has not enough space
#TMPDIR=/opt/zapret/tmp

# redefine user for zapret daemons. required on Keenetic
#WS_USER=nobody

# override firewall type : iptables,nftables,ipfw
FWTYPE=iptables

# options for ipsets
# maximum number of elements in sets. also used for nft sets
SET_MAXELEM=522288
# too low hashsize can cause memory allocation errors on low RAM systems , even if RAM is enough
# too large hashsize will waste lots of RAM
IPSET_OPT="hashsize 262144 maxelem $SET_MAXELEM"
# dynamically generate additional ip. $1 = ipset/nfset/table name
#IPSET_HOOK="/etc/zapret.ipset.hook"

# options for ip2net. "-4" or "-6" auto added by ipset create script
IP2NET_OPT4="--prefix-length=22-30 --v4-threshold=3/4"
IP2NET_OPT6="--prefix-length=56-64 --v6-threshold=5"
# options for auto hostlist
AUTOHOSTLIST_RETRANS_THRESHOLD=3
AUTOHOSTLIST_FAIL_THRESHOLD=3
AUTOHOSTLIST_FAIL_TIME=60
# 1 = debug autohostlist positives to ipset/zapret-hosts-auto-debug.log
AUTOHOSTLIST_DEBUGLOG=0

# number of parallel threads for domain list resolves
MDIG_THREADS=30

# ipset/*.sh can compress large lists
GZIP_LISTS=1
# command to reload ip/host lists after update
# comment or leave empty for auto backend selection : ipset or ipfw if present
# on BSD systems with PF no auto reloading happens. you must provide your own command
# set to "-" to disable reload
#LISTS_RELOAD="pfctl -f /etc/pf.conf"

# override ports
#HTTP_PORTS=80-81,85
#HTTPS_PORTS=443,500-501
#QUIC_PORTS=443,444

# CHOOSE OPERATION MODE
# MODE : nfqws,tpws,tpws-socks,filter,custom
# nfqws : nfqws for dpi desync
# tpws : tpws transparent mode
# tpws-socks : tpws socks mode
# filter : no daemon, just create ipset or download hostlist
# custom : custom mode. should modify custom init script and add your own code
MODE=nfqws
# apply fooling to http
MODE_HTTP=1
# for nfqws only. support http keep alives. enable only if DPI checks for http request in any outgoing packet
MODE_HTTP_KEEPALIVE=0
# apply fooling to https
MODE_HTTPS=1
# apply fooling to quic
#MODE_QUIC=1
# none,ipset,hostlist,autohostlist
MODE_FILTER=hostlist

# CHOOSE NFQWS DAEMON OPTIONS for DPI desync mode. run "nfq/nfqws --help" for option list
DESYNC_MARK=0x40000000
DESYNC_MARK_POSTNAT=0x20000000
NFQWS_OPT_DESYNC="--dpi-desync=fake,split2 --dpi-desync-ttl=3 --dpi-desync-fooling=md5sig"
#NFQWS_OPT_DESYNC_HTTP="--dpi-desync=split --dpi-desync-ttl=0 --dpi-desync-fooling=badsum"
#NFQWS_OPT_DESYNC_HTTPS="--wssize=1:6 --dpi-desync=split --dpi-desync-ttl=0 --dpi-desync-fooling=badsum"
#NFQWS_OPT_DESYNC_HTTP6="--dpi-desync=split --dpi-desync-ttl=5 --dpi-desync-fooling=none"
#NFQWS_OPT_DESYNC_HTTPS6="--wssize=1:6 --dpi-desync=split --dpi-desync-ttl=5 --dpi-desync-fooling=none"
#NFQWS_OPT_DESYNC_QUIC="--dpi-desync=fake --dpi-desync-repeats=6"
#NFQWS_OPT_DESYNC_QUIC6="--dpi-desync=hopbyhop"

# CHOOSE TPWS DAEMON OPTIONS. run "tpws/tpws --help" for option list
TPWS_OPT="--hostspell=HOST --split-http-req=method --split-pos=3 --oob"

# openwrt only : donttouch,none,software,hardware
FLOWOFFLOAD=donttouch

# openwrt: specify networks to be treated as LAN. default is "lan"
#OPENWRT_LAN="lan lan2 lan3"
# openwrt: specify networks to be treated as WAN. default wans are interfaces with default route
#OPENWRT_WAN4="wan vpn"
#OPENWRT_WAN6="wan6 vpn6"

# for routers based on desktop linux and macos. has no effect in openwrt.
# CHOOSE LAN and optinally WAN/WAN6 NETWORK INTERFACES
# or leave them commented if its not router
# it's possible to specify multiple interfaces like this : IFACE_LAN="eth0 eth1 eth2"
# if IFACE_WAN6 is not defined it take the value of IFACE_WAN
#IFACE_LAN=eth0
#IFACE_WAN=eth1
#IFACE_WAN6="ipsec0 wireguard0 he_net"

# should start/stop command of init scripts apply firewall rules ?
# not applicable to openwrt with firewall3+iptables
INIT_APPLY_FW=1
# firewall apply hooks
#INIT_FW_PRE_UP_HOOK="/etc/firewall.zapret.hook.pre_up"
#INIT_FW_POST_UP_HOOK="/etc/firewall.zapret.hook.post_up"
#INIT_FW_PRE_DOWN_HOOK="/etc/firewall.zapret.hook.pre_down"
#INIT_FW_POST_DOWN_HOOK="/etc/firewall.zapret.hook.post_down"

# do not work with ipv4
#DISABLE_IPV4=1
# do not work with ipv6
#DISABLE_IPV6=1

# select which init script will be used to get ip or host list
# possible values : get_user.sh get_antizapret.sh get_combined.sh get_reestr.sh get_hostlist.sh
# comment if not required
GETLIST=get_antizapret_domains.sh

Собран curl с библиотекой GnuTLS

./curlgnutls -V
curl 8.10.0 (x86_64-pc-linux-gnu) libcurl/8.10.0 GnuTLS/3.8.7 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libpsl/0.21.5 nghttp2/1.63.0 librtmp/2.3 OpenLDAP/2.6.8
Release-Date: 2024-09-11
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets zst

вывод при использовании tlsv1.2

./curlgnutls -vIS --tlsv1.2 --tls-max 1.2 https://files.catbox.moe
* Host files.catbox.moe:443 was resolved.
* IPv6: (none)
* IPv4: 108.181.20.37
*   Trying 108.181.20.37:443...
* Connected to files.catbox.moe () port 443
* GnuTLS ciphers: NORMAL:-ARCFOUR-128:-CTYPE-ALL:+CTYPE-X509:-VERS-SSL3.0:-VERS-TLS-ALL:+VERS-TLS1.2
* found 152 certificates in /etc/ssl/certs/ca-certificates.crt
* found 760 certificates in /etc/ssl/certs
* SSL connection using TLS1.2 / ECDHE_ECDSA_CHACHA20_POLY1305
*   server certificate verification OK
*   server certificate status verification SKIPPED
*   common name: catbox.moe (matched)
*   server certificate expiration date OK
*   server certificate activation date OK
*   certificate public key: EC/ECDSA
*   certificate version: #3
*   subject: CN=catbox.moe
*   start date: Mon, 26 Aug 2024 05:23:58 GMT
*   expire date: Sun, 24 Nov 2024 05:23:57 GMT
*   issuer: C=US,O=Let's Encrypt,CN=E6
* ALPN: server accepted h2
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://files.catbox.moe/
* [HTTP/2] [1] [:method: HEAD]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: files.catbox.moe]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.10.0]
* [HTTP/2] [1] [accept: */*]
> HEAD / HTTP/2
> Host: files.catbox.moe
> User-Agent: curl/8.10.0
> Accept: */*
> 
* Request completely sent off
< HTTP/2 301 
HTTP/2 301 
< server: nginx
server: nginx
< date: Wed, 18 Sep 2024 01:10:02 GMT
date: Wed, 18 Sep 2024 01:10:02 GMT
< content-type: text/html; charset=UTF-8
content-type: text/html; charset=UTF-8
< location: https://catbox.moe
location: https://catbox.moe
< strict-transport-security: max-age=31536000; includeSubDomains; preload
strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-content-type-options: nosniff
x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< x-frame-options: DENY
x-frame-options: DENY
< content-security-policy: default-src 'self'; img-src 'self' https://quickchart.io https://files.catbox.moe; media-src 'self' https://files.catbox.moe; style-src 'self' 'unsafe-inline'; script-src https://www.google.com https://www.gstatic.com 'self' 'unsafe-inline'; frame-src https://www.google.com;
content-security-policy: default-src 'self'; img-src 'self' https://quickchart.io https://files.catbox.moe; media-src 'self' https://files.catbox.moe; style-src 'self' 'unsafe-inline'; script-src https://www.google.com https://www.gstatic.com 'self' 'unsafe-inline'; frame-src https://www.google.com;
< referrer-policy: same-origin
referrer-policy: same-origin
< permissions-policy: accelerometer=(),ambient-light-sensor=(),autoplay=(),battery=(),camera=(),display-capture=(),document-domain=(),encrypted-media=(),execution-while-not-rendered=(),execution-while-out-of-viewport=(),fullscreen=(),gamepad=(),geolocation=(),gyroscope=(),hid=(),identity-credentials-get=(),idle-detection=(),local-fonts=(),magnetometer=(),microphone=(),midi=(),otp-credentials=(),payment=(),picture-in-picture=(),publickey-credentials-create=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),speaker-selection=(),storage-access=(),usb=(),web-share=(),xr-spatial-tracking=()
permissions-policy: accelerometer=(),ambient-light-sensor=(),autoplay=(),battery=(),camera=(),display-capture=(),document-domain=(),encrypted-media=(),execution-while-not-rendered=(),execution-while-out-of-viewport=(),fullscreen=(),gamepad=(),geolocation=(),gyroscope=(),hid=(),identity-credentials-get=(),idle-detection=(),local-fonts=(),magnetometer=(),microphone=(),midi=(),otp-credentials=(),payment=(),picture-in-picture=(),publickey-credentials-create=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),speaker-selection=(),storage-access=(),usb=(),web-share=(),xr-spatial-tracking=()
< 

* Connection #0 to host files.catbox.moe left intact

С tlsv1.3

./curlgnutls -vIS --tlsv1.3 --tls-max 1.3 https://files.catbox.moe
* Host files.catbox.moe:443 was resolved.
* IPv6: (none)
* IPv4: 108.181.20.37
*   Trying 108.181.20.37:443...
* Connected to files.catbox.moe () port 443
* GnuTLS ciphers: NORMAL:-ARCFOUR-128:-CTYPE-ALL:+CTYPE-X509:-VERS-SSL3.0:-VERS-TLS-ALL:+VERS-TLS1.3
* found 152 certificates in /etc/ssl/certs/ca-certificates.crt
* found 760 certificates in /etc/ssl/certs
* Recv failure: Соединение разорвано другой стороной
* closing connection #0
curl: (56) Recv failure: Соединение разорвано другой стороной

curl собранный с openssl работает как по tlsv1.2, так и 1.3

curl 8.10.0 (x86_64-pc-linux-gnu) libcurl/8.10.0 OpenSSL/3.3.2 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.63.0 nghttp3/1.5.0
Release-Date: 2024-09-11
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

OpenSSL с tlsv1.2

curl -vIS --tlsv1.2 --tls-max 1.2 https://files.catbox.moe
* Host files.catbox.moe:443 was resolved.
* IPv6: (none)
* IPv4: 108.181.20.37
*   Trying 108.181.20.37:443...
* Connected to files.catbox.moe () port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305 / secp384r1 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=catbox.moe
*  start date: Aug 26 05:23:58 2024 GMT
*  expire date: Nov 24 05:23:57 2024 GMT
*  subjectAltName: host "files.catbox.moe" matched cert's "*.catbox.moe"
*  issuer: C=US; O=Let's Encrypt; CN=E6
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://files.catbox.moe/
* [HTTP/2] [1] [:method: HEAD]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: files.catbox.moe]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.10.0]
* [HTTP/2] [1] [accept: */*]
> HEAD / HTTP/2
> Host: files.catbox.moe
> User-Agent: curl/8.10.0
> Accept: */*
> 
* Request completely sent off
< HTTP/2 301 
HTTP/2 301 
< server: nginx
server: nginx
< date: Wed, 18 Sep 2024 01:12:49 GMT
date: Wed, 18 Sep 2024 01:12:49 GMT
< content-type: text/html; charset=UTF-8
content-type: text/html; charset=UTF-8
< location: https://catbox.moe
location: https://catbox.moe
< strict-transport-security: max-age=31536000; includeSubDomains; preload
strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-content-type-options: nosniff
x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< x-frame-options: DENY
x-frame-options: DENY
< content-security-policy: default-src 'self'; img-src 'self' https://quickchart.io https://files.catbox.moe; media-src 'self' https://files.catbox.moe; style-src 'self' 'unsafe-inline'; script-src https://www.google.com https://www.gstatic.com 'self' 'unsafe-inline'; frame-src https://www.google.com;
content-security-policy: default-src 'self'; img-src 'self' https://quickchart.io https://files.catbox.moe; media-src 'self' https://files.catbox.moe; style-src 'self' 'unsafe-inline'; script-src https://www.google.com https://www.gstatic.com 'self' 'unsafe-inline'; frame-src https://www.google.com;
< referrer-policy: same-origin
referrer-policy: same-origin
< permissions-policy: accelerometer=(),ambient-light-sensor=(),autoplay=(),battery=(),camera=(),display-capture=(),document-domain=(),encrypted-media=(),execution-while-not-rendered=(),execution-while-out-of-viewport=(),fullscreen=(),gamepad=(),geolocation=(),gyroscope=(),hid=(),identity-credentials-get=(),idle-detection=(),local-fonts=(),magnetometer=(),microphone=(),midi=(),otp-credentials=(),payment=(),picture-in-picture=(),publickey-credentials-create=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),speaker-selection=(),storage-access=(),usb=(),web-share=(),xr-spatial-tracking=()
permissions-policy: accelerometer=(),ambient-light-sensor=(),autoplay=(),battery=(),camera=(),display-capture=(),document-domain=(),encrypted-media=(),execution-while-not-rendered=(),execution-while-out-of-viewport=(),fullscreen=(),gamepad=(),geolocation=(),gyroscope=(),hid=(),identity-credentials-get=(),idle-detection=(),local-fonts=(),magnetometer=(),microphone=(),midi=(),otp-credentials=(),payment=(),picture-in-picture=(),publickey-credentials-create=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),speaker-selection=(),storage-access=(),usb=(),web-share=(),xr-spatial-tracking=()
< 

* Connection #0 to host files.catbox.moe left intact

OpenSSL с tlsv1.3

curl -vIS --tlsv1.3 --tls-max 1.3 https://files.catbox.moe
* Host files.catbox.moe:443 was resolved.
* IPv6: (none)
* IPv4: 108.181.20.37
*   Trying 108.181.20.37:443...
* Connected to files.catbox.moe () port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / secp384r1 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=catbox.moe
*  start date: Aug 26 05:23:58 2024 GMT
*  expire date: Nov 24 05:23:57 2024 GMT
*  subjectAltName: host "files.catbox.moe" matched cert's "*.catbox.moe"
*  issuer: C=US; O=Let's Encrypt; CN=E6
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://files.catbox.moe/
* [HTTP/2] [1] [:method: HEAD]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: files.catbox.moe]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.10.0]
* [HTTP/2] [1] [accept: */*]
> HEAD / HTTP/2
> Host: files.catbox.moe
> User-Agent: curl/8.10.0
> Accept: */*
> 
* Request completely sent off
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 301 
HTTP/2 301 
< server: nginx
server: nginx
< date: Wed, 18 Sep 2024 01:14:03 GMT
date: Wed, 18 Sep 2024 01:14:03 GMT
< content-type: text/html; charset=UTF-8
content-type: text/html; charset=UTF-8
< location: https://catbox.moe
location: https://catbox.moe
< strict-transport-security: max-age=31536000; includeSubDomains; preload
strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-content-type-options: nosniff
x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< x-frame-options: DENY
x-frame-options: DENY
< content-security-policy: default-src 'self'; img-src 'self' https://quickchart.io https://files.catbox.moe; media-src 'self' https://files.catbox.moe; style-src 'self' 'unsafe-inline'; script-src https://www.google.com https://www.gstatic.com 'self' 'unsafe-inline'; frame-src https://www.google.com;
content-security-policy: default-src 'self'; img-src 'self' https://quickchart.io https://files.catbox.moe; media-src 'self' https://files.catbox.moe; style-src 'self' 'unsafe-inline'; script-src https://www.google.com https://www.gstatic.com 'self' 'unsafe-inline'; frame-src https://www.google.com;
< referrer-policy: same-origin
referrer-policy: same-origin
< permissions-policy: accelerometer=(),ambient-light-sensor=(),autoplay=(),battery=(),camera=(),display-capture=(),document-domain=(),encrypted-media=(),execution-while-not-rendered=(),execution-while-out-of-viewport=(),fullscreen=(),gamepad=(),geolocation=(),gyroscope=(),hid=(),identity-credentials-get=(),idle-detection=(),local-fonts=(),magnetometer=(),microphone=(),midi=(),otp-credentials=(),payment=(),picture-in-picture=(),publickey-credentials-create=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),speaker-selection=(),storage-access=(),usb=(),web-share=(),xr-spatial-tracking=()
permissions-policy: accelerometer=(),ambient-light-sensor=(),autoplay=(),battery=(),camera=(),display-capture=(),document-domain=(),encrypted-media=(),execution-while-not-rendered=(),execution-while-out-of-viewport=(),fullscreen=(),gamepad=(),geolocation=(),gyroscope=(),hid=(),identity-credentials-get=(),idle-detection=(),local-fonts=(),magnetometer=(),microphone=(),midi=(),otp-credentials=(),payment=(),picture-in-picture=(),publickey-credentials-create=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),speaker-selection=(),storage-access=(),usb=(),web-share=(),xr-spatial-tracking=()
< 

* Connection #0 to host files.catbox.moe left intact

Из за этой проблемы невозможно использовать wget с files.catbox.moe.

wget https://files.catbox.moe
--2024-09-18 05:15:24--  https://files.catbox.moe/
Загружен сертификат CA «/etc/ssl/certs/ca-certificates.crt»
Распознаётся files.catbox.moe (files.catbox.moe)… 108.181.20.37
Подключение к files.catbox.moe (files.catbox.moe)|108.181.20.37|:443... соединение установлено.
GnuTLS: Error in the pull function.
Не удаётся установить SSL-соединение.
bol-van commented 3 days ago

На других заблокированных ресурсах тоже самое ? Пробовали gnutls через нецензурируемое соединение (VPN) ?