lay295 / TwitchDownloader

Twitch VOD/Clip Downloader - Chat Download/Render/Replay
MIT License
2.59k stars 260 forks source link

VOD Download problem: The SSL connection could not be established, see inner exception. #934

Closed lchenIXL closed 7 months ago

lchenIXL commented 7 months ago

Checklist

Edition

Both

Describe your issue here

Using TwitchDownloaderCLI-1.53.9-Windows-x64, I tried the following: TwitchDownloaderCLI VideoDownload --id 1971552437 -o output.mp4

I got the following error message:

TwitchDownloaderCLI 1.53.9 Copyright (c) lay295 and contributors
[STATUS] - Fetching Video Info [1/5]Unhandled exception. System.AggregateExcepti
on: One or more errors occurred. (The SSL connection could not be established, s
ee inner exception.)
 ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
 ---> System.ComponentModel.Win32Exception (0x80090326): The message received was unexpected or badly formatted.
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter , Boolean , Byte[] , Boolean )
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions , HttpRequestMessage , Boolean , Stream , CancellationToken )
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions , HttpRequestMessage , Boolean , Stream , CancellationToken )
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage )
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken )
   at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(H
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage , HttpCompletionOption , CancellationTokenSource , Boolean , CancellationTokenSource , CancellationToken )
   at TwitchDownloaderCore.TwitchHelper.GetVideoPlaylist(Int32 videoId, String token, String sig) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCore/TwitchHelper.cs:line 65
   at TwitchDownloaderCore.VideoDownloader.GetQualityPlaylist(GqlVideoResponse videoInfo) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCore/VideoDownloader.cs:line 568
   at TwitchDownloaderCore.VideoDownloader.DownloadAsync(CancellationToken cancellationToken) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCore/VideoDownloader.cs:line 59
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean )
   at System.Threading.Tasks.Task.Wait(Int32 , CancellationToken )
   at System.Threading.Tasks.Task.Wait()
   at TwitchDownloaderCLI.Modes.DownloadVideo.Download(VideoDownloadArgs inputOptions) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCLI/Modes/DownloadVideo.cs:line 23
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at TwitchDownloaderCLI.Program.Main(String[] args) in /home/runner/work/TwitchDownloader/TwitchDownloader/TwitchDownloaderCLI/Program.cs:line 33

TwitchDownloaderGUI-1.53.9-Windows-x64 also gives the error: ERROR: The SSL connection could not be established, see inner exception.

But chat download for both TwitchDownloaderCLI-1.53.9-Windows-x64 and TwitchDownloaderGUI-1.53.9-Windows-x64 was successful.

Add any related files or extra information here

I'm on Windows 7 64-bit. But this had never caused any problems before.

lay295 commented 7 months ago

Interesting read about this problem here if you're interested, the TLDR of it is basically to stop using Windows 7 since it doesn't have the cyphers needed to communicate with the webserver twitch is using.

I assume we can bundle our own cyphers or something, will have to get a Windows 7 machine/vm to test though.

EDIT: Er maybe not? I don't see an easy way to resolve this really looking around.

lay295 commented 7 months ago

Man, I hate the solution but I think I'll just set up a reverse proxy for this case (if we catch a nested AuthenticaionException). I don't really want to lose support for Windows 7, but this might just be a temporary fix because the might update their GQL endpoint to also only support newer cyphers potentially.

Any ideas from anybody else?

image

ScrubN commented 7 months ago

but this might just be a temporary fix because the might update their GQL endpoint to also only support newer cyphers potentially.

Over the last few months, Twitch has been making fairly large API changes. I wouldn't be surprised in the GQL endpoint gets the new cyphers requirement in upcoming weeks, meaning either we should switch to an IHttpClientFactory with Polly to automatically retry with the reverse proxy (so we don't have to copy-paste code for every single request), or just stop supporting windows 7.

Edit:

switch to an IHttpClientFactory with Polly to automatically retry with the reverse proxy

This means refactoring the entire project to use services with dependency injection instead of static helper classes. I have been wanting to do this anyways to prevent socket starvation, but it's still just so much work.

XRayXI commented 7 months ago

On my end, also a Win7 64 bits machine*** the latest version can not grab the ID of a VOD (screenshot). I used the older version 1.53.6 just fine until the morning. Even though I had the Downloader.exe set in my firewall not to check for updates and let the program released in the firewall and could start the download. Version 1.53.6 and that latest one, does not grab/download any VODs now.

I understand the switch, that I might need to take to 'give up' on Win7, but that machine is my server for anything and I basically do not care about the risks involved at Win7. It is not connected to my inner network and storage is outside the box ;-)

The problem that I have right now is time, to get Win10 f.e. on that machine. Time that is what I only have two days to complete that and that I do not have, and the VODs will be gone. That channel has 2 months of VODs and deleting them over that time. Wishing that I could do this Windows upgrades next weekend.

/Edit: The 'only' problem with using the older one, is that the selection of Quality is not working. Tried with Task Queue too, but that fails after a few errors with 'Error'. That was not a problem last night.

Screenshot_3834

ScrubN commented 7 months ago

The problem that I have right now is time, to get Win10 f.e. on that machine. Time that is what I only have two days to complete that and that I do not have, and the VODs will be gone. That channel has 2 months of VODs and deleting them over that time. Wishing that I could do this Windows upgrades next weekend.

If it is not too much of a hassle, I would suggest using TwitchDownloaderCLI in a linux livecd environment. My suggestion is to download manjaro gnome, use rufus to create a livecd on a usb drive, and boot your server to that. After that, you can find CLI run instructions in the readme. You should be able to mount and the internal drives in the server via the Disks software, and you should set the --temp-path and --output arguments to the internal drives if the usb drive is under 64GB.

If you are unable to use TwitchDownloaderCLI, you can try twitch-downloader-gui from the AUR, as manjaro is arch-based. You may need to use yum to install it.

/Edit: The 'only' problem with using the older one, is that the selection of Quality is not working. Tried with Task Queue too, but that fails after a few errors with 'Error'. That was not a problem last night.

That is because Twitch changed their security requirements of the usher. Previously, we were (unintentionally) calling it via http, however they changed that endpoint to require security certificates that are not part of windows 7, meaning the insecure method we were using before no longer works anywhere.

XRayXI commented 7 months ago

Thank you very, very much! I will go with your tips tomorrow and try Linux first. I tried several other tools, but this one is my favorite. With Linux this machine(6700TE CPU) can have a DUAL Boot.

There is always something that makes you move hardware around, and other things in your way haha. It will be Yum for sure ;-)

Again, thank you!

lay295 commented 7 months ago

Over the last few months, Twitch has been making fairly large API changes. I wouldn't be surprised in the GQL endpoint gets the new cyphers requirement in upcoming weeks, meaning either we should switch to an IHttpClientFactory with Polly to automatically retry with the reverse proxy (so we don't have to copy-paste code for every single request), or just stop supporting windows 7.

I just put in a reverse proxy using Cloudflare workers as a workaround. If they ever do the same on the GQL endpoint I say we probably just drop support for Windows 7 at that point. I set up a Win 7 VM and seems to work fine, so can release an update if you want to.

ScrubN commented 7 months ago

1.54.0 has been released with several fixes. Here's hoping it's the last hotfix :p

XRayXI commented 7 months ago

1.54.0

Version 1.54.0 confirmed working for/at OS Win7 64bits. ID (full Twitch link) accepted, Quality could be set (720/60) and downloading now. Thank you so much and having a thumbs up for the work!

If anything I'd wish for, the GUI "Task Queue" (top right blue box) to be able to set the Quality or I am missing out a setting. Right now the Task is set at default the max quality automated by the queue. But, I am very happy (due to time/upgrade OS) that I can use it again. This 'server' is going to be busy for a few days ;-)

ScrubN commented 7 months ago

If anything I'd wish for, the GUI "Task Queue" (top right blue box) to be able to set the Quality or I am missing out a setting. Right now the Task is set at default the max quality automated by the queue.

I know, and I don't particularly like how limited the mass downloader options are either, but maintaining 2 different download pages also sucks. In an effort to help with this, I added a right click menu to the VOD/Clip search windows with an option to copy video IDs, so you can search for your VOD and download it with the dedicated page without needing to use a web browser.

In the future I would like to get rid of the dedicated download pages altogether so there is only the task queue and a rich download options window, but that's a bit of an ambitious goal that won't be done in a weekend.