msys2 / MINGW-packages

Package scripts for MinGW-w64 targets to build under MSYS2.
https://packages.msys2.org
BSD 3-Clause "New" or "Revised" License
2.25k stars 1.21k forks source link

GnuTLS still not working properly? #14739

Open asctime opened 1 year ago

asctime commented 1 year ago

Description / Steps to reproduce the issue

gnutls 3.7 backported. Using something like this simple tls connection test: gnutls-cli -d 1 imap.gmail.com -p 993

My understanding is that the windows store issues were resolved, or do they still exist? Can anyone verify on latest? Or is this just me? Alt store workarounds? Thanks..

Expected behavior (tested on recent Linux box)

gnutls-cli -d 1 imap.gmail.com -p 993 Processed 137 CA certificate(s). Resolving 'imap.gmail.com:993'... Connecting to '74.125.68.109:993'...

Actual behavior (MinGW64 version, backported and recompiled)

X:\>gnutls-cli -d 1 imap.gmail.com -p 993 |<1>| There was a non-CA certificate in the trusted list: C=XX,ST=XX,L=XX =XXX |<1>| There was a non-CA certificate in the trusted list: OU=Copyright (c) 1997 Microsoft Corp.,OU=Microsoft Corporation,CN=Microsoft Root Authority. |<1>| There was a non-CA certificate in the trusted list: C=US,O=MSFT,CN=Microso ft Authenticode(tm) Root Authority. |<1>| There was a non-CA certificate in the trusted list: CN= ... (lots more of that line and finally..) Processed 988 CA certificate(s). Resolving 'imap.gmail.com:993'... **Connecting to '172.253.118.108:993'... |<1>| Discarded message[0] due to invalid decryption * Fatal error: Decryption has failed.

Windows Version

10 Pro

MINGW environments affected

Biswa96 commented 1 year ago

Thank you for reporting this issue. I can reproduce the issue in my system.

Click here to see the backtrace of the segfault ```c Thread 1 received signal SIGSEGV, Segmentation fault. 0x00007ff729d516da in system_write (ptr=0x3, data=0x243dcaf96cb, data_size=396) at ../../gnutls-3.7.8/src/common.h:128 128 return send(hd->fd, data, data_size, 0); (gdb) bt #0 0x00007ff729d516da in system_write (ptr=0x3, data=0x243dcaf96cb, data_size=396) at ../../gnutls-3.7.8/src/common.h:128 #1 0x00007ffbc7165c59 in _gnutls_writev_emu (session=0x243dcaf45f0, fd=0x3, giovec=0xfe0dffd0b0, giovec_cnt=1, vec=0) at ../../gnutls-3.7.8/lib/buffers.c:450 #2 0x00007ffbc7165e4d in _gnutls_writev (session=0x243dcaf45f0, giovec=0xfe0dffd0b0, giovec_cnt=1, total=396) at ../../gnutls-3.7.8/lib/buffers.c:506 #3 0x00007ffbc7166622 in _gnutls_io_write_flush (session=0x243dcaf45f0) at ../../gnutls-3.7.8/lib/buffers.c:700 #4 0x00007ffbc7166db0 in _gnutls_handshake_io_write_flush (session=0x243dcaf45f0) at ../../gnutls-3.7.8/lib/buffers.c:839 #5 0x00007ffbc716da0d in _gnutls_send_handshake2 (session=0x243dcaf45f0, bufel=0x243dcaf8350, type=GNUTLS_HANDSHAKE_CLIENT_HELLO, queue_only=0) at ../../gnutls-3.7.8/lib/handshake.c:1450 #6 0x00007ffbc716d4ed in _gnutls_send_handshake (session=0x243dcaf45f0, bufel=0x243dcaf8350, type=GNUTLS_HANDSHAKE_CLIENT_HELLO) at ../../gnutls-3.7.8/lib/handshake.c:1287 #7 0x00007ffbc7171026 in send_client_hello (session=0x243dcaf45f0, again=0) at ../../gnutls-3.7.8/lib/handshake.c:2357 #8 0x00007ffbc7172a05 in handshake_client (session=0x243dcaf45f0) at ../../gnutls-3.7.8/lib/handshake.c:3052 #9 0x00007ffbc717248c in gnutls_handshake (session=0x243dcaf45f0) at ../../gnutls-3.7.8/lib/handshake.c:2884 #10 0x00007ff729d555bd in do_handshake (rpl_socket=0xfe0dffeb20) at ../../gnutls-3.7.8/src/cli.c:1855 #11 0x00007ff729d5c79c in socket_open2 (hd=0xfe0dffeb20, hostname=0x243dad15290 "imap.gmail.com", service=0x7ff729d85080 "993", app_proto=0x0, flags=64, msg=0x7ff729d7893b "Connecting to", rdata=0x0, edata=0x0, server_trace=0x0, client_trace=0x0) at ../../gnutls-3.7.8/src/socket.c:602 #12 0x00007ff729d54347 in main (argc=6, argv=0x243dad02df0) at ../../gnutls-3.7.8/src/cli.c:1371 ```
asctime commented 1 year ago

Then be aware this issue in GnuTLS likely has far reaching affects on the entire distro. Any MinGW64 package currently using GnuTLS-certificate-store-based web communication will probably fail. For example, ffmpeg:

ffplay https://www.bloomberg.com/media-manifest/streams/eu.m3u8 Compiled currently --enable-gnutls:

tls @ 0000000004f17700] Decryption has failed.KB sq=    0B f=0/0
**ttps://www.bloomberg.com/media-manifest/streams/eu.m3u8: I/O error**
   nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0

.. which in turn can cause QmlAVDemuxer Unable to open input file: "I/O error" (-5) when that build of ffmpeg is used in Qt-QML or some other place. etc..

. I changed my TLS backend to openssl --enable-openssl (which IMO has always worked much more reliably on Windows from the very start, anyway):

[hls,applehttp @ 0000000004698fc0] Opening 'https://liveprodeuwest.akamaized.net
/eu1/...
(successfully plays)

Someone probably should double-check every dependent package, unless a complete, reliable and thoroughly tested GnuTLS fix for all Windows versions is quickly found (again, unlikely but possible now that we see the scale of the problem in real-world live testing..)

Biswa96 commented 1 year ago

After reverting the src/common.h hunk from this commit https://gitlab.com/gnutls/gnutls/-/commit/20f993aca3c08a779cd350bf2093d01a6309a32e, the program is working as expected.

I have reported the issue in upstream here https://gitlab.com/gnutls/gnutls/-/issues/1446

asctime commented 1 year ago

Not for me :( I refactored all immediate dependencies to their latest versions, reversed the chunk, but it made no difference in the outcome for me. I also tested the updated ca-certificate store to bypass the Windows system store, which does eliminate all those hundreds of noisy "non-CA-certificate" warnings but does not resolve the bigger decryption issue:

X:\>gnutls-cli -d 1 imap.gmail.com -p 993 --x509cafile=/MSYS2/mingw64/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem --disable-extensions
Processed 127 CA certificate(s).
Resolving 'imap.gmail.com:993'...
Connecting to '74.125.130.108:993'...
|<1>| Discarded message[0] due to invalid decryption
*** Fatal error: Decryption has failed.

Also it should be noted: I never got a segfault. I'm not sure how or why you did, but the answer is probably slightly different something or other.. In any case, unresolved, here.

Biswa96 commented 1 year ago

I get this message *** Fatal error: Error in the certificate. Try to remove the /MSYS2 string and path should start with /mingw64.

asctime commented 1 year ago

@Biswa96 are you running the command in native shell or bash?

Biswa96 commented 1 year ago

bash in mintty.

asctime commented 1 year ago

@Biswa96 that makes sense for an MSYS app but the entire idea of MinGW is native Windows apps runnign in a native WIndows environment without the Cygwin pseudo-posix stuff. Please try it in cmd.

Biswa96 commented 1 year ago

Did you change PATH variable in cmd? What is the PWD?

asctime commented 1 year ago

Yes, my path has been modified for both the general bin as well as the system compiler, with MSYS binaries playing second fiddle. As they should.

In my case, PWD is /, but I don't see how that's really relevant. Please set up your systems paths as typical MinGW user would and try it.

Biswa96 commented 1 year ago

OK, I have tried it in CMD and got same error as previous *** Fatal error: Error in the certificate.

I have set the PATH with this

set PATH=F:\msys64\ucrt64\bin;%PATH%

Then run the command like this

.\bin\gnutls-cli -d 1 imap.gmail.com -p 993 --x509cafile=F:\msys64\ucrt64\etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem --disable-extensions

The PWD is the install directory after compiling gnutls with reverting the src/common.h hunk.

asctime commented 1 year ago

I'm not sure how that could be correct if you are using MSYS2-project defaults. Here is my recommended path for MinGW64 users who chose the MSYS2 distro:

Path=X:\B;X:\B\bin;X:\MSYS2\mingw64\bin;X:\MSYS2\mingw64\libexec;X:\MSYS2\mingw64\x86_64-w64-mingw32\bin;X:\MSYS2\usr\bin;C:\Program Files (x86)\Common...

Some explanation:

  1. All paths are REAL paths. Win32 doesn't properly support softlinks, nodelinks or anything else posix.
  2. MSYS is positioned BEHIND MinGW, so MinGW always gets priority. Per my very good discussion with @revelator many moons ago, the art is always in properly understanding the mix of MInGW and MSYS.
  3. "X:\B" is a master batchfile folder. I recommend every programmer have one, and only batchfile scripts with TOP PRIORITY should be placed there.

Not just for you but since the topic came up I think it's time to share..

Edit: I decided to very edit that copy-and-paste of my actual path to remove the Kotlin-Java, Enrgyplus, etc as it could (unbelievably but true) cause confusion. Sorry for the edited version isn't precisely real but it will help other readers focus on the point here..

asctime commented 1 year ago

Yeah it's still unusable for me. For example, pidgin won't connect to jabber:

...
(17:57:59) gnutls: Starting handshake with xmpp.xxxx
(17:57:59) gnutls: Handshake failed. Error Decryption has failed.
...

Same error. I'll try to do some more digging now that I've further isolated this: 1) My certificate store isn't solely the issue (see above) 2) Nothing looks missing when I do a simple 'gnutls-cli -l' and compare that output to a linux box.

So I might ask, can anyone here connect to their xmpp account using the latest MinGW64 pidgin binaries and gnutls libraries?

revelator commented 1 year ago

looks like multiple people have trouble with it especially when connecting to HTTPS or other secure protocols, one hint i got was --disable-tls13-interop.

asctime commented 1 year ago

@revelator thanks, didn't fix it but got me looking. What I came up with is: --with-default-priority-string='NORMAL:-VERS-TLS1.3'

That doesn't solve the issue but it does work around it. For now. While I was at it, I set up something practical for a MinGW configuration: --with-system-priority-file=/MSYS2/mingw64/etc/gnutls/default-priorities

and also added switching to our own certificate store to cut down the noise until that's fixed: --with-default-trust-store-file=/MSYS2/mingw64/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

It is working a bit better:

X:\>gnutls-cli google.com
Processed 127 CA certificate(s).
Resolving 'google.com:443'...
Connecting to '142.250.204.14:443'...
- Certificate type: X.509
- Got a certificate list of 3 certificates.
...

I still get connection errors but things go much further than before. Does anyone want to confirm?

BTW Pidgin still doesn't connect to xmpp maybe it defines TLS-1.3 internally?

revelator commented 1 year ago

hmm a few oddities i noticed was that gnutls-cli would allways segfault in mintty but works just fine in the cmd shell, i dont have pidgin installed so cannot test that atm. running strace on the command gave me a heap of cygwin internals being used but did not really tell me much about why it crashes in mintty, guess i need a debug build.

asctime commented 1 year ago

@revelator Don't really get the mintty talk for MinGW. The whole apparent MSYS2 axiom, that everything for windows should work inside a somewhat dubious posix emulator kind of make my head spin like "wait.. what?". Trying to make everything work like that will always break somewhere - and for MinGW it should way sooner break in mintty and not on the desktop, IMO. MinGW64 mintty support probably isn't something a lot of users need or would even understand. Besides, there's a native MSYS2 version of gnutls - just use that in mintty. Just saying..

revelator commented 1 year ago

oh i get that definatly x-) funny though the idea with the whole emulated posix shell was originally to make it easier for windows users with a little knowledge of posix commands to setup and compile libraries that where originally not intended for windows. strangely over time it grew into this -> (monstously complicated thing) which in itself is pretty cool if you are a power user but can easily make less inclined people ditch it in disgust.

revelator commented 1 year ago

short version of above if it is compiled for windows then yeah it should absolutely run in cmd ;-) not nessesarily in bash or whatever posix shell though it would be nice for debugging purposes.

asctime commented 1 year ago

funny though the idea with the whole emulated posix shell was originally to make it easier for windows users with a little knowledge of posix commands to setup and compile libraries that where originally not intended for windows.

Yup which is why msys was sub-folder of mingw32, and not the other way around. Krosis we digress ;)

PURPLE_GNUTLS_DEBUG=9 pidgin -d confirms the issue to still be a GnuTLS-issue, not an NSS one:

(17:01:58) gnutls: lvl 5: REC[00000000090ecd30]: SSL 63.120 Unknown Packet packe
t received. Epoch 0, length: 28012
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/record.c[check_recv_typ
e]:634
(17:01:58) gnutls: lvl 1: Received record packet of unknown type 60
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/record.c[recv_headers]:
1228
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/record.c[_gnutls_recv_i
n_buffers]:1321
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/buffers.c[_gnutls_hands
hake_io_recv_int]:1467
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/handshake.c[_gnutls_rec
v_handshake]:1600
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/handshake.c[handshake_c
lient]:3075
(17:01:58) gnutls: Handshake failed. Error An unexpected TLS packet was received
.
(17:01:58) connection: Connection error on 00000000091be470 (reason: 5 descripti
on: SSL Handshake Failed)
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/buffers.c[_gnutls_io_wr
ite_flush]:696
(17:01:58) gnutls: lvl 5: REC: Sending Alert[1|0] - Close notify
(17:01:58) gnutls: lvl 5: REC[00000000090ecd30]: Preparing Packet Alert(21) with
 length: 2 and min pad: 0
(17:01:58) gnutls: lvl 9: ENC[00000000090ecd30]: cipher: NULL, MAC: MAC-NULL, Ep
och: 0
(17:01:58) gnutls: lvl 5: REC[00000000090ecd30]: Sent Packet[2] Alert(21) in epo
ch 0 and length: 7
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/record.c[check_session_
status]:1661
(17:01:58) gnutls: lvl 3: ASSERT: ../../gnutls-3.7.8/lib/record.c[gnutls_bye]:33
5
(17:01:58) gnutls: lvl 5: REC[00000000090ecd30]: Start of epoch cleanup
(17:01:58) gnutls: lvl 5: REC[00000000090ecd30]: End of epoch cleanup
(17:01:58) gnutls: lvl 5: REC[00000000090ecd30]: Epoch #0 freed
asctime commented 1 year ago

This fixes GnuTLS for me, the cli interface now connects reliably so I am posting this for review and confirmation by others.

The big problem was indeed the certificate store. You really need to be using the right one and the default WIndows store doesn't seem to work correctly for MinGW. And pulling individual files from p11-kit seemed like a clever old school way of doing things but is also wrong. TLDR; I think we need to change PKGCONFIG to add the following configuration(x2) line at the very least:

--with-default-trust-store-pkcs11="pkcs11:"

It works on most Linux distros and seems to work for MinGW too. My cli connections are very reliable now. To prove this to myself I also no longer need the TLS-1.3 work-round I posted earlier. Just change to p11-kit as the default store and everything else just works!

The second part of the problem we caused ourselves by reversing an upstream patch that probably shouldn't be reversed. I changed common.h back to it's original state before the chunk reversal and it solved the remaining cli issues for me. Anyone want to try it out? Afterwards maybe @Biswa96 depending on confirmation you might want to call off the dogs upstream.. It was all us apparently just faulty packaging and lack of actual testing.

PS pidgin still doesn't work right but with cli working perfectly I'm starting to believe there's an implementation/configuration issue I'll keep looking. But with finally working gnutls tools I think we can eliminate that dependency from the equation..

revelator commented 1 year ago

aye disregard the NSS comment it was fixed upstream :) but great that gnutls works now.

revelator commented 1 year ago

strange as it might seem this also fixes the msys2 gnutls, it was segfaulting on every adress i tried untill i recompiled it with these options ->

    --with-default-trust-store-pkcs11="pkcs11:" \
    --with-default-trust-store-file=/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
asctime commented 1 year ago

@revelator that's actually not supposed to work: you can only have one trust ca at a time. I think there's a patch now to prevent this misconception that you can use both. Anyway it's obviously using one or the other - which is obviously better that neither!

revelator commented 1 year ago

ah ok, well it does work now so maybe something that needs looking at for the msys2 version ?.

asctime commented 1 year ago

I think what you will find is simply one of those stores are being used and one isn't. But it's one of those cases of too much information is still heaps better that too little. In other words: at least now you are giving it options to work with. Here is an example from the busybox guys:

http://lists.busybox.net/pipermail/buildroot/2018-November/235973.html

revelator commented 1 year ago

was set to one of the openssl certs previously but i suspect the format from those are wrong for gnutls ? i tried removing either of them and it still works as long as it uses the pkcs11 store.

revelator commented 1 year ago

was --with-default-trust-store-file=/usr/ssl/certs/ca-bundle.crt before, format looks a bit the same (except the suffix .pem vs .crt) so im not totally sure if this is what broke it or just that gnutls needed to be rebuild becuse of some updated dependency

asctime commented 1 year ago

OK found a new problem: the TLS re-handshake negation actually never takes place. There's no error but epoch 0 simply ends after the first HELLO, always, even with -e is specified. Are you getting that as well?

revelator commented 1 year ago

yup :/

asctime commented 1 year ago

Well it's pretty obvious now from our comparisons and lack of any other feedback that this actually never worked properly under MinGW64 and possibly has broader issues under Windows. E.g. this guy starts with the same Windows issue I posted, and the tail still hangs loose: https://gitlab.com/gnutls/gnutls/-/issues/1365

I'll keep chipping away at it when I have time tho.

revelator commented 1 year ago

damn ok ill try out a few things as well maybe we can get it seaworthy :)

asctime commented 1 year ago

I have left comments upstream: https://gitlab.com/gnutls/gnutls/-/issues/1446#note_1238421936 https://gitlab.com/gnutls/gnutls/-/issues/1365#note_1238469543

asctime commented 1 year ago

latest MinGW64-shared tests results:

============================================================================
Testsuite summary for GnuTLS 3.7.8
============================================================================
# TOTAL: 375
# PASS:  233
# SKIP:  130
# XFAIL: 0
# FAIL:  12
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
Please report to bugs@gnutls.org
============================================================================

FAIL: mini-eagain-dtls.exe FAIL: x509cert-ct.exe <-------!!! FAIL: mini-emsgsize-dtls.exe FAIL: dtls-handshake-versions.exe FAIL: dtls-max-record.exe FAIL: handshake-false-start.exe <-------!!! FAIL: version-checks.exe <-------!!! FAIL: dtls12-cert-key-exchange.exe FAIL: dtls10-cert-key-exchange.exe FAIL: dtls-repro-20170915.exe FAIL: dtls1-2-mtu-check.exe FAIL: trust-store.exe <-------!!!

dependencies: brotli = 1.0.9 (all checks pass) libidl2 = 0.8.14-1 "This package does not come with a test suite. "

============================================================================
Testsuite summary for GNU Libtasn1 4.19.0
============================================================================
# TOTAL: 31
# PASS:  31
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

nettle = 3.7.3

====================
All 109 tests passed
====================

p11-kit = 0.24. memcheck test requires requires valgrind :(

============================================================================
Testsuite summary     [ for libunistring 0.9.10-2 ]
============================================================================
# TOTAL: 507
# PASS:  490
# SKIP:  13
# XFAIL: 0
# FAIL:  4
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
============================================================================

Note libunistring test suite has obviously never been patched for MinGW. Did surprisingly well: Needed to REMOVE TESTS test-pthread_sigmask1 & test-pthread_sigmask1 (No POSIX signal masking in Windows) FAIL: test-mbrtowc5.sh (PASS test-mbrtowc-w32-5.sh passes (exit status: 0)) FAIL: test-init.sh (not ported: failed test: mkdir: cannot change permissions) FAIL: test-verify.sh (not ported: failed test: mkdir: cannot change permissions) FAIL: test-xalloc-die.sh (not ported: failed test: mkdir: cannot change permissions) .. These are all shell script issues not API failures, later on we might consider patching-porting but OK for now .

@revelator can you post your shared make-check just for MinGW64-gnutls when you have a moment please?

revelator commented 1 year ago

sure but give me sec, got pneumonia :S so im not totally upto speed (sleeping most of the time and coughing my lungs up yuck).

asctime commented 1 year ago

New upstream tickets: https://gitlab.com/gnutls/gnutls/-/issues/1449 https://gitlab.com/gnutls/gnutls/-/issues/1450

Biswa96 commented 1 year ago

Upstream has added a fix with this commit https://gitlab.com/gnutls/gnutls/-/commit/4a065f3fddf1cf77a531a90aec85a41293b009d2. Now the gnutls-cli -d 1 imap.gmail.com -p 993 command does not show segfault. Would you like to test any other commands?

asctime commented 1 year ago

@Biswa96 yes he posted that a few days ago. It doesn't fix any of the issues which I have opened. Also please remember that I was unable to duplicate segfault to start with. My tickets have nothing to do with a segfault. I am guessing that your issue is also UCRT related. As we saw, I don't use UCRT. As I asked @revelator, please run the gnutls test suite and post your results, thank you.

Biswa96 commented 1 year ago

See this build log after adding the patch file https://github.com/Biswa96/MINGW-packages/actions/runs/4110599015

asctime commented 1 year ago

Can you? From what I see the checks are not even run..

Biswa96 commented 1 year ago

See the full build log carefully. Here is the test result of shared library in ucrt64

2023-02-07T05:27:34.3735505Z ============================================================================
2023-02-07T05:27:34.3736961Z Testsuite summary for GnuTLS 3.7.8
2023-02-07T05:27:34.3737443Z ============================================================================
2023-02-07T05:27:34.3737744Z # TOTAL: 157
2023-02-07T05:27:34.3738066Z # PASS:  139
2023-02-07T05:27:34.3738324Z # SKIP:  6
2023-02-07T05:27:34.3738998Z # XFAIL: 0
2023-02-07T05:27:34.3740060Z # FAIL:  12
2023-02-07T05:27:34.3740382Z # XPASS: 0
2023-02-07T05:27:34.3741583Z # ERROR: 0
asctime commented 1 year ago

Thanks. These are the same results I posted. Original issues seem to remain, still pending further comment from upstream. Correct?

Biswa96 commented 1 year ago

The failed tests seem to be from gnulib functions, not from gnutls. By origin issue, do you mean the command with --disable-extensions option?

asctime commented 1 year ago

I think I found the log file to which you refer - not the summary display but rather download the zipped archive from the appropriate architecture build. Yes, this is not the proper gnutls "make check", only the gnulib tests. The fact that the same number of tests failed is simply pure coincidence 😮 It still appears to me that cloud build is not running the proper gnutls test suite at all (it takes a long time, and does it once for static and once for shared - - I still don't see that being done anywhere... should I be looking at another log file?

asctime commented 1 year ago

@Biswa96 about original issue, yes and no -- as you read the thread you see that the problem grew in scope which is why I decided to run the entire test suite manually and post results. Basically, gnutls still does not connect properly, nor do any dependent applications such as PIDGIN, FFMPEG or anything else requiring gnutls for secure connection. The gnutls-cli example is just for simple demonstration purposes.

Biswa96 commented 1 year ago

...any dependent applications such as PIDGIN, FFMPEG or https://github.com/msys2/MINGW-packages/issues/14739#issuecomment-1371831217 requiring gnutls

I have tried the ffplay https://www.bloomberg.com/media-manifest/streams/eu.m3u8 command as you suggested. It works without any issue and even without changing anything in mingw gnutls package.

BTW, the video is boring and everyone is talking about money 😃

asctime commented 1 year ago

@Biswa96 I spent some time on this, please read my following comments carefully as there are several points to consider, some of this I already posted before but let's try again:

First .. No I am still unable to use gnutls connections at all. Just gnutls, everything else like openssl works fine for me. Also my pidgin logs are very clear that there is a gnutls connection issue. My guess is that it has to do with a TLS re-handshake error and I am trying to diagnose it. One upstream ticket that I opened revolves around datagram tls failures. I don't know if it's related. No one has commented.

Second .. I posted the ffmpeg comment over a month ago, and you are just checking it now .. 🦀 OK I notice in cloud build that ipv6 is enabled and this could be a factor for those with ipv6 deactivated on the network. Or maybe MINTTY has some certstore settings that cover the problem. Just a passing thought rn.

Third .. I rebuilt the very latest MINGW-packages/gnutls (today's) without difficulties and I am still getting the same results on my end. For example, I am curious that you also do not seem to be seeing any windows certificate store issues when you run, like I get on my screen massive (100s) amounts of this kind of thing:

|<1>| There was a non-CA certificate in the trusted list: CN=<hostname>.
|<1>| There was a non-CA certificate in the trusted list: CN=<hostname>. 

You don't get that? Now, I do have a couple of localCA certificates in my Windows store, a lot of people do. But certainly not enough to generate so many warnings like that.. Other people have reported similar results on Windows, so my way of working around this was to use the --with-default-trust-store-pkcs11 option, which stops the CA-certificate spam above but does not solve the connection issue (first problem).

Forth .. The test suite results which you posted are actually from a completely separate GNULIB set of tests, which make runs automatically but presents as GNUTLS results (I believe this to a packaging error) All of this is just gnulib:

2023-02-07T05:20:28.5200119Z PASS: test-vasnprintf.exe
2023-02-07T05:20:28.6141290Z PASS: test-vasprintf.exe
2023-02-07T05:20:28.7083910Z PASS: test-verify.exe
2023-02-07T05:20:29.4244221Z PASS: test-usleep.exe
2023-02-07T05:20:29.4319702Z FAIL: test-verify.sh
2023-02-07T05:20:29.5323101Z PASS: test-vsnprintf.exe
2023-02-07T05:20:29.5556471Z PASS: test-wchar.exe
2023-02-07T05:20:30.2592355Z FAIL: test-xalloc-die.sh
2023-02-07T05:20:30.5109751Z PASS: test-stat-time.exe
2023-02-07T05:20:31.1753577Z ============================================================================
2023-02-07T05:20:31.1754633Z Testsuite summary for GnuTLS 3.7.8
2023-02-07T05:20:31.1754993Z ============================================================================
2023-02-07T05:20:31.1755283Z # TOTAL: 157
2023-02-07T05:20:31.1755707Z # PASS:  139
2023-02-07T05:20:31.1755938Z # SKIP:  6
2023-02-07T05:20:31.1756160Z # XFAIL: 0
2023-02-07T05:20:31.1756337Z # FAIL:  12
2023-02-07T05:20:31.1756562Z # XPASS: 0
2023-02-07T05:20:31.1756775Z # ERROR: 0
2023-02-07T05:20:31.1757560Z ============================================================================

The GNUTLS test suite from "make check" does not seem to be getting run, probably because of the confusion with the gnulib test suite. I would like to see the actual gnutls test results please (tests like x509cert-ct.exe, dtls-handshake-versions.exe, trust-store.exe, etc).

Fifth .. I tried to compare my config.log with information from cloud build, but the format is not the same and conclusions may be incorrect. How can I get the intermediate files from your cloud build, so that I can compare if there is something on my system which may be causing issues? Ultimately I am just trying to gather enough information to enable solving the first problem for myself, but it seems really hard to get..

revelator commented 1 year ago

sorry havent had time to look further into it yet, first i got sick now and now my old developer machine finally bit the dust after 12 years of faithfull service (had to happen someday) but it had a good run. So the last few days i been installing new hardware and now i have to recreate my build environment before i can give it another go... i hope to be up and running again soon though.