search

rockdaboot / wget2

The successor of GNU Wget. Contributions preferred at https://gitlab.com/gnuwget/wget2. But accepted here as well 😍
GNU Lesser General Public License v3.0
557 stars 76 forks source link

When compiled using GnuTLS, HTTP2 does not work #266

Closed catharsis71 closed 2 years ago

catharsis71 commented 2 years ago

I am running Ubuntu WSL 22.04.1 LTS

I compiled with --with-ssl=gnutls as otherwise I have too many problems, but compiling this way introduced an additional problem

Now HTTP2 does not work, which it did before

$ wget2_noinstall --http2=on -d https://www.google.com/
~/.wget2rc is deprecated. Please move it to /home/XXXXXXXXX/.config/wget/wget2rc
08.132647.347 Reading /home/XXXXXXXXX/.wget2rc
08.132647.347 name=tcp-fastopen value=0 invert=0
08.132647.348 name=http2 value=on invert=0
08.132647.348 Local URI encoding = 'ANSI_X3.4-1968'
08.132647.348 Input URI encoding = 'ANSI_X3.4-1968'
08.132647.351 Fetched HSTS data from '/home/XXXXXXXXX/.local/share/wget/.wget-hsts'
08.132647.351 Fetched HPKP data from '/home/XXXXXXXXX/.local/share/wget/.wget-hpkp'
08.132647.351 Fetched OCSP hosts from '/home/XXXXXXXXX/.local/share/wget/.wget-ocsp_hosts'
08.132647.351 Fetched OCSP fingerprints from '/home/XXXXXXXXX/.local/share/wget/.wget-ocsp'
08.132647.352 set_exit_status(0)
08.132647.352 *url =
08.132647.352 *3 https://www.google.com/
08.132647.352 transcoded 0 bytes from 'utf-8' to 'ANSI_X3.4-1968'
08.132647.352 local filename = 'index.html'
08.132647.352 host_add_job: job fname index.html
08.132647.352 host_add_job: 0x7fffe50695a0 https://www.google.com/
08.132647.352 host_add_job: qsize 1 host-qsize=1
08.132647.352 queue_size: qsize=1
08.132647.352 queue_size: qsize=1
08.132647.352 queue_size: qsize=1
08.132647.352 [0] action=1 pending=0 host=0x0
08.132647.353 dequeue job https://www.google.com/
08.132647.353 resolving www.google.com:443...
08.132647.391 has 2607:f8b0:4000:806::2004:443
08.132647.391 has 142.251.45.36:443
08.132647.391 trying 2607:f8b0:4000:806::2004:443...
08.132647.391 Failed to set socket option TCP_FASTOPEN_CONNECT
08.132647.391 GnuTLS init
08.132647.402 GnuTLS system certificate store is empty
08.132647.402 Certificates loaded: 120
08.132647.402 GnuTLS init done
08.132647.402 TLS False Start requested
WARNING: OCSP is not available in this version of GnuTLS.
08.132647.436 host has no pubkey pinnings stored in hpkp db
08.132647.437 host has no pubkey pinnings stored in hpkp db
08.132647.437 host has no pubkey pinnings stored in hpkp db
08.132647.437 host has no pubkey pinnings stored in hpkp db
08.132647.437 host has no pubkey pinnings stored in hpkp db
08.132647.437 host has no pubkey pinnings stored in hpkp db
08.132647.437 TLS False Start: off
----
Certificate info [0]:
  Valid since: Mon Sep 12 03:19:33 2022
  Expires: Mon Dec  5 02:19:32 2022
  Fingerprint: 6542b71fc440764b1324abcc86a633d4
  Serial number: 6542b71fc440764b1324abcc86a633d4
  Public key: EC/ECDSA, High (256 bits)
  Version: #3
  DN: CN=www.google.com
  Issuer's DN: C=US,O=Google Trust Services LLC,CN=GTS CA 1C3
  Issuer's OID: 2.5.4.6
  Issuer's UID: 2.5.4.6
Certificate info [1]:
  Valid since: Wed Aug 12 19:00:42 2020
  Expires: Wed Sep 29 19:00:42 2027
  Fingerprint: 178ef18343ccc9e0ecb0e38d9dea03d8
  Serial number: 178ef18343ccc9e0ecb0e38d9dea03d8
  Public key: RSA, Medium (2048 bits)
  Version: #3
  DN: C=US,O=Google Trust Services LLC,CN=GTS CA 1C3
  Issuer's DN: C=US,O=Google Trust Services LLC,CN=GTS Root R1
  Issuer's OID: 2.5.4.6
  Issuer's UID: 2.5.4.6
Certificate info [2]:
  Valid since: Thu Jun 18 19:00:42 2020
  Expires: Thu Jan 27 18:00:42 2028
  Fingerprint: 3682b6c0eb81959e4b4458dfbb65d4f7
  Public key: RSA, High (4096 bits)
  Version: #3
  DN: C=US,O=Google Trust Services LLC,CN=GTS Root R1
  Issuer's DN: C=BE,O=GlobalSign nv-sa,OU=Root CA,CN=GlobalSign Root CA
  Issuer's OID: 2.5.4.6
  Issuer's UID: 2.5.4.6
----
Ephemeral ECDH using curve X25519
Key Exchange: ECDHE-RSA
Protocol: TLS1.3
Certificate Type: X.509
Cipher: AES-256-GCM
MAC: AEAD
----
08.132647.439 Handshake completed
08.132647.439 established connection www.google.com
[0] Downloading 'https://www.google.com/' ...
08.132647.439 cookie_create_request_header for host=www.google.com path=
08.132647.439 # sent 194 bytes:
GET / HTTP/1.1
Host: www.google.com
Accept-Encoding: gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: wget2/2.0.1
Connection: keep-alive

08.132647.439 [0] action=2 pending=1 host=0x7fffe5069520
08.132647.439 ### req 0x7f35a02c2590 pending requests = 1
08.132647.512 # got header 1056 bytes:
HTTP/1.1 200 OK
Date: Sat, 08 Oct 2022 18:26:48 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Content-Encoding: gzip
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2022-10-08-18; expires=Mon, 07-Nov-2022 18:26:48 GMT; path=/; domain=.google.com; Secure
Set-Cookie: AEC=AakniGOfC5p72ZAT0dgfUGftlV8f3ItyS8MPq7LWTMi0hiA0ZQTeF0koVQ; expires=Thu, 06-Apr-2023 18:26:48 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax
Set-Cookie: NID=511=twYFEjNCT0fPk5hb-R5vC2-e2jR4-yXT4HUD7I0HfTy-zpG3bODrzu1cq2PpYX5ehrnBYmqSFcgGA9sLvKVmnPlCWLKBsYN-3k57NQ3OTnD4UboCfRZoVxZ1y41Ja5_7r5plX_z7_TFKIvwotnAAqVkq98J_rJMlvS8kG4TYwOI; expires=Sun, 09-Apr-2023 18:26:48 GMT; path=/; domain=.google.com; HttpOnly
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Transfer-Encoding: chunked

08.132647.513 Unsupported cookie-av 'SameSite'
Saving 'index.html.16'
08.132647.514 blacklist set filename: index.html -> index.html.16
08.132647.514 method 1 138 0:
08.132647.514 need at least 5018 more bytes
08.132647.514 chunk completed
HTTP response 200 OK [https://www.google.com/]
08.132647.515 keep_alive=1
08.132647.515 domain_match(google.com,www.google.com)
08.132647.515 domain_match(google.com,www.google.com)
08.132647.515 domain_match(google.com,www.google.com)
08.132647.515 got cookie 1P_JAR=2022-10-08-18
08.132647.515 store new cookie 1P_JAR=2022-10-08-18
08.132647.515 got cookie AEC=AakniGOfC5p72ZAT0dgfUGftlV8f3ItyS8MPq7LWTMi0hiA0ZQTeF0koVQ
08.132647.515 store new cookie AEC=AakniGOfC5p72ZAT0dgfUGftlV8f3ItyS8MPq7LWTMi0hiA0ZQTeF0koVQ
08.132647.515 got cookie NID=511=twYFEjNCT0fPk5hb-R5vC2-e2jR4-yXT4HUD7I0HfTy-zpG3bODrzu1cq2PpYX5ehrnBYmqSFcgGA9sLvKVmnPlCWLKBsYN-3k57NQ3OTnD4UboCfRZoVxZ1y41Ja5_7r5plX_z7_TFKIvwotnAAqVkq98J_rJMlvS8kG4TYwOI
08.132647.515 store new cookie NID=511=twYFEjNCT0fPk5hb-R5vC2-e2jR4-yXT4HUD7I0HfTy-zpG3bODrzu1cq2PpYX5ehrnBYmqSFcgGA9sLvKVmnPlCWLKBsYN-3k57NQ3OTnD4UboCfRZoVxZ1y41Ja5_7r5plX_z7_TFKIvwotnAAqVkq98J_rJMlvS8kG4TYwOI
08.132647.515 _host_remove_job: 0x7fffe50695a0
08.132647.515 host_remove_job: qsize=0 host->qsize=0
08.132647.515 [0] action=1 pending=0 host=0x7fffe5069520
08.132647.515 closing connection
08.132647.516 [0] action=1 pending=0 host=0x0
08.132647.516 main: wake up
08.132647.516 main: done
08.132647.517 Successfully updated '/home/XXXXXXXXX/.local/share/wget/.wget-ocsp_hosts'.
08.132647.518 Saved OCSP hosts to '/home/XXXXXXXXX/.local/share/wget/.wget-ocsp_hosts'
08.132647.519 Successfully updated '/home/XXXXXXXXX/.local/share/wget/.wget-ocsp'.
08.132647.519 Saved OCSP fingerprints to '/home/XXXXXXXXX/.local/share/wget/.wget-ocsp'
08.132647.519 blacklist https://www.google.com/

I have these GnuTLS packages installed:

$ apt list --installed | grep gnutls

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

gnutls-bin/jammy-updates,jammy-security,now 3.7.3-4ubuntu1.1 amd64 [installed,automatic]
libcurl3-gnutls/jammy-updates,jammy-security,now 7.81.0-1ubuntu1.3 amd64 [installed,automatic]
libgnutls-dane0/jammy-updates,jammy-security,now 3.7.3-4ubuntu1.1 amd64 [installed,automatic]
libgnutls-openssl27/jammy-updates,jammy-security,now 3.7.3-4ubuntu1.1 amd64 [installed,auto-removable]
libgnutls28-dev/jammy-updates,jammy-security,now 3.7.3-4ubuntu1.1 amd64 [installed,auto-removable]
libgnutls30/jammy-updates,jammy-security,now 3.7.3-4ubuntu1.1 amd64 [installed,automatic]
libgnutlsxx28/jammy-updates,jammy-security,now 3.7.3-4ubuntu1.1 amd64 [installed,auto-removable]
libsrt1.4-gnutls/jammy,now 1.4.4-4 amd64 [installed,automatic]
catharsis71 commented 2 years ago

Actually I'm not sure that something is going wrong with the compiling...

I deleted all the versions I compiled previously and started from scratch, compiling 3 different copies, for OpenSSL, GnuTLS, and WolfSSL, renamed accordingly so I can track which is which

I also still have the 1.99 Ubuntu package installed

The --version on the 1.99 looks like this:

GNU Wget2 1.99.1 - multithreaded metalink/file/website downloader

+digest +https +ssl/gnutls +ipv6 +iri +large-file +nls -ntlm -opie +psl +iconv +idn2 +zlib +lzma +brotlidec +bzip2 +http2 +gpgme

I see there's a "+http2"

but my compiled versions all show "-http2"

my compiled GnuTLS version also shows "-ssl" instead of "+ssl/gnutls", however, SSL definitely does work with it

$ wget2_gnutls --version
~/.wget2rc is deprecated. Please move it to /home/cmcphers/.config/wget/wget2rc
GNU Wget2 2.0.1 - multithreaded metalink/file/website downloader

+digest -https -ssl +ipv6 +iri +large-file +nls -ntlm -opie -psl -hsts
+iconv -idn +zlib -lzma -brotlidec -zstd -bzip2 -lzip -http2 -gpgme

Copyright (C) 2012-2015 Tim Ruehsen
Copyright (C) 2015-2021 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please send bug reports and questions to <bug-wget@gnu.org>.

$ wget2_openssl --version
~/.wget2rc is deprecated. Please move it to /home/cmcphers/.config/wget/wget2rc
GNU Wget2 2.0.1 - multithreaded metalink/file/website downloader

+digest +https +ssl/openssl +ipv6 +iri +large-file +nls -ntlm -opie -psl -hsts
+iconv -idn +zlib -lzma -brotlidec -zstd -bzip2 -lzip -http2 -gpgme

Copyright (C) 2012-2015 Tim Ruehsen
Copyright (C) 2015-2021 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please send bug reports and questions to <bug-wget@gnu.org>.

$ wget2_wolfssl --version
~/.wget2rc is deprecated. Please move it to /home/cmcphers/.config/wget/wget2rc
GNU Wget2 2.0.1 - multithreaded metalink/file/website downloader

+digest +https +ssl/wolfssl +ipv6 +iri +large-file +nls -ntlm -opie -psl -hsts
+iconv -idn +zlib -lzma -brotlidec -zstd -bzip2 -lzip -http2 -gpgme

Copyright (C) 2012-2015 Tim Ruehsen
Copyright (C) 2015-2021 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please send bug reports and questions to <bug-wget@gnu.org>.
catharsis71 commented 2 years ago

Disregard, I had libnghttp2-14 installed but not libnghttp2-dev