bug,crash

[javlock]

os: Linux archlinux 5.15.11-arch2-1 #1 SMP PREEMPT Wed, 22 Dec 2021 09:23:54 +0000 x86_64 GNU/Linux ssl: openssl 1.1.1.m-1

wget2 --version: GNU Wget2 2.0.0

coredump: 1.zip

[rockdaboot]

Thanks for you report.

I cannot reproduce on Debian Unstable with OpenSSL 1.1.1l-1 with wget2 --progress bar --max-threads 10 https://www.vatican.va/ -r -k -p -E (command line from the core dump).

Did you press CTRL-C ? In this case a core dump is expected. This has changed in latest git master, though.

Please reopen if you still think this is an unfixed bug and when you have more information, e.g. a fully symbolized backtrace or the binary that was used for the core dump (including debug symbols).

[javlock]

@rockdaboot ctrl c didn't press UDP:the situation repeated itself the message in the console before the system "Crash (memory stack flushed to disk)" is: "double free or corruption"

1)the downloaded program was delivered without debug symbols, tell me how to build with them (I download the repository), (I write in java, I don’t know how to set up program builders for c/c++/c# (make, cmake, etc.)) 2)I know that I can specify a prefix for building a project, I don't remember what to write

1.debug symbols 2.dir for build&&install

[javlock]

i will try this CCFLAGS=-ggdb ./configure --prefix=/home/user/wget2crash/build

[rockdaboot]

How to build+install wget2 from tarball, see https://gitlab.com/gnuwget/wget2/#downloading-and-building-from-tarball

In your case I suggest a slightly different set of commands

wget https://gnuwget.gitlab.io/wget2/wget2-latest.tar.gz
tar xf wget2-latest.tar.gz
cd wget2-*
CFLAGS="-g -O0" ./configure -with-ssl=openssl
make -j($nproc)
make -j($nproc) check

You need to install at least some of the dependencies first, like the openssl development files.

Then you use the src/wget2_noinstall binary - it doesn't require installing anything.

I don't have a ready-to-go Dockerfile, but you might have a look into the Arch Dockerfile from our CI: https://gitlab.com/gnuwget/build-images/-/tree/master/docker-archlinux

[javlock]

arch linux uses ...-1.16 all automake utilities: installed 16.needed 15

[rockdaboot]

Hm, aclocal 1.16 works fine here and is detected. I don't think you need to care when building from tarball, btw.

[javlock]

there was an incomprehensible error with dependencies, now it was assembled normally started the download

[javlock]

the assembled version does not crash

[javlock]

based on the names of the functions on the stack, the error in ssl when received by the issuer. mk not every launch of wget2 it crashes, I can assume that this is due to the intervention of state structures, we have a mitm organized in our country

[rockdaboot]

So does this issue happen to you with both OpenSSL and GnuTLS ? Not sure, if I understand this correctly.

For GnuTLS I'd say that every gnutls_free() automatically sets the free'd pointer to NULL. So a second free is just a no-op. Except the pointer has been copied or a structure wasn't initialized to 0 and then realloc was used.

Even with a (weird) MITM, nothing should crash.

[javlock]

CFLAGS="-g -O0" ./configure-with-ssl=openssl` to build with gnutls you need to remove-with-ssl=openssl` or specify another? UDP:I found in the configure help that gnutls by default, removed the argument

about mitm: I live in Russia, here at the state level mitm

[javlock]

another question, are you managing the AUR package wget2-git ?

[rockdaboot]

No, I have no experience with Arch or Arch packaging. Maybe it is @darnir !?

[javlock]

I cannot send the document ...

[javlock]

coredump https://github.com/javlock/wtf-docs/raw/main/wget2/wget2-gnutls.zip

[javlock]

#8  0x00007f033afb1bf8 in verify_certificate_callback (session=0x7f03213707a0) at ssl_gnutls.c:947
        status = 0x2
        deinit_cert = 0x0
        deinit_issuer = 0x0
        cert_list = 0x0
        cert_list_size = 0x0
        ret = 0xffffffff
        err = 0x7f03
        ocsp_ok = 0x0
        pinning_ok = 0x0
        cert = 0x0
        issuer = 0x0
        hostname = 0x7f0321b6e580 "www.vatican.va"
        tag = 0x7f033afeff05 "ERROR"
        nvalid = 0x0
        nrevoked = 0x0
        nignored = 0x0
        ctx = 0x7f0321b467f0

#8  0x00007f033afb1bf8 in verify_certificate_callback (session=0x7f03213707a0) at ssl_gnutls.c:947
947     if (gnutls_certificate_verify_peers3(session, hostname, &status) != GNUTLS_E_SUCCESS) {

[rockdaboot]

Could you also upload the wget2 binary with that you created the core dump ?

[rockdaboot]

What we know so far is that you get a TLS rehandshake which triggers a bug either in wget2 or in GnuTLS. While the coredump + wget2 binary give us everything at the moment of the crash, we don't know how to reproduce (e.g. it could be a chain of rehandshakes that triggers the bug).

It could be helpful to record the network flow with wireshark, tcpdump or tshark. It might be some amount of data, not sure if there is a limit when uploading to github. Having the flow of data on the TLS layer, I (or someone at GnuTLS) could create a reproducer and fix the issue.

You are using GnuTLS 3.7.2 ?

If there really is a try of MITM, I assume that it needs to trigger some TLS bugs/flaws in versions < TLS1.2.

[rockdaboot]

Maybe a low-hanging fruit worth testing (before wireshark etc): GNUTLS_DEBUG_LEVEL=9 in front of your wget2 command line (better leave --progress bar away and add -q). If possible, upload the compressed output to your github project.

[rockdaboot]

In the logs I saw that for every request a new connection (incl. TLS handshake) was made. Just pushed a fix for that (keep-alive is default on HTTP/1.1).

That is not really related to this bug - but as the number of TLS handshakes is largely reduced, you might not see the crash any more. Please test.

[javlock]

here is a link to gnutls 3.7.2-2 which I have, soon after the update this version will become unavailable (rolling release) download link

dir with build wget I upload* as build.zip with gnutls in build/GNUTLS/

I'll start launching with debug args now

[javlock]

state we cut the speed 83 KiB

[javlock]

you can unpack .zst files ? do you have a zstd utility?

[rockdaboot]

Sure, will unpack it with zstd tomorrow

[javlock]

i downloaded the logs and coredumps with gnutls debug

[javlock]

I download traffic dump, keys for decoding* and coredump wget2/logs/tls