Use `CookieContainer` to handle cookies between requests

[xBaank]

By using a CookieContainer, we can manage cookies between requests more effectively, which allows us to download private videos.

[xBaank]

If we want to use a CookieContainer there is a problem, we can't create a new instance of the CookieContainer in the HttpMessageHandler once a request is made and we can't clear the cookies from it, so if an user login with an account and then login with another account we can't clear the cookies, and creating a new container throws an exception if a request is made.

[xBaank]

This is the only solution I found, set the Cookie as expired to "clear" it from the CookieContainer.

[Tyrrrz]

Hey @xBaank, thanks! I gave it a test and still couldn't pull a private video, it said the video iss unavailable.

Can you let me know how you tested it so I can repeat the same process? Thanks 🙂

[xBaank]

Sure!

1. Go to settings and press login
2. Login with your google credentials
3. If login success you get redirected to www.youtube.com, cookies get extracted and embedded browser is closed.
4. Insert a private video and you should be able to download it.

You can see here how I get the video is unavailable while Im not logged, but once I login I can download the video perfectly.

https://github.com/Tyrrrz/YoutubeDownloader/assets/21276786/dda1603e-15ff-4390-9034-ee115389e2ca

[xBaank]

I tested with 2 different accounts and I can download private videos, idk what we are missing :thinking: Maybe this was the problem? https://github.com/yt-dlp/yt-dlp/blob/49296437a8e5fa91dacb5446e51ab588474c85d3/yt_dlp/extractor/youtube.py#L573-L590 https://github.com/yt-dlp/yt-dlp/issues/393#issuecomment-858228198

[Tyrrrz]

Hmm, that's weird, it doesn't appear to work with the private video I was testing (I tried the latest build). I wonder if it has something to do with account settings?

Here are the cookies it pulled:

Does your account use 2FA?

[xBaank]

One account has 2FA, but the one I used in that video doesn't have it.

[Tyrrrz]

Do you have the same set of cookies as I do?

[xBaank]

No, I don't have VISITOR_INFO1_LIVE and you don't have __SECURE-YEC:

[Tyrrrz]

No, I don't have VISITOR_INFO1_LIVE and you don't have __SECURE-YEC:

Hmm. If you remove __SECURE-YEC from there, does it affect your ability to download private videos?

[xBaank]

No, I can still download the videos. Does it change something if you remove VISITOR_INFO1_LIVE ?

[Tyrrrz]

Nope, still the same. @derech1e mentioned this on Discord:

(included link)

However, the video in question is uploaded by my personal account, which is the same one I authenticated with.

[Tyrrrz]

That said, I was able to load my "watch later" playlist correctly, so I guess the cookies were enough for that 🤔

I'm still curious if there's anything obvious we might be missing regarding that private video. But I'm okay with merging this for now and iterating later. I want to clean up the code a bit anyway.

Does that sound good @xBaank?

[xBaank]

I think I found something, with my google account I created a second youtube channel, before creating the second one I was able to download watch later playlist and private videos but after creating that second channel Im not able to download the videos of any channel but Im able to download the watch later playlist of the first channel. Do you have multiple channels in that google account as well? BTW feel free to merge 😄.

[xBaank]

I think I managed to fix the problem in this commit (It is on a different branch just in case you want merge what we already have). Now if you have multiple channels on one account you are able to select the channel, The problem was that if it is a secondary channel we need the SyncId/PageId. This was mentioned in yt-dlp source code

datasyncid is of the form "channel_syncid||user_syncid" for secondary channel and just "user_syncid||" for primary channel. We only want the channel_syncid Extract syncId required to download private playlists of secondary channels

I tested with main and secondary channel of same google account and I was able to download a private video on both channels.

[Tyrrrz]

Do you have multiple channels in that google account as well?

Yes, I do, actually.

I think I managed to fix the problem in this commit (It is on a different branch just in case you want merge what we already have). Now if you have multiple channels on one account you are able to select the channel, The problem was that if it is a secondary channel we need the SyncId/PageId.

This sounds like it should be it! Can you append these changes to this PR? I will test the CI build and then merge them together.

[Tyrrrz]

I tried the latest build but unfortunately I still can't pull the private video :/

I did end up deleting the brand account, so now it's just my personal account. Now I'm getting this error:

I'm assuming it might be because the newly added request to https://www.youtube.com/getDatasyncIdsEndpoint fails if you only have one account?

[xBaank]

Idk why it fails 😓, I tested with a new account that have only one channel and with an account that have multiple channels (Brand account) and it doesn't fail when sending that request. And the only thing needed for that request seems to be the cookies that you get in www.youtube.com. This is the request:

GET /getDatasyncIdsEndpoint HTTP/1.1
Host: www.youtube.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate, br
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Connection: keep-alive
Cookie: __Secure-YEC=...; CONSENT=PENDING+632; PREF=f4=4000000&f6=40000000&tz=Europe.Madrid&f7=100&f5=20000; SOCS=...; __Secure-3PAPISID=...; __Secure-3PSID=...; __Secure-3PSIDCC=...; HSID=...; SSID=...; APISID=...; SAPISID=...; __Secure-1PAPISID=...; SID=...; __Secure-1PSID=...; LOGIN_INFO=...; SIDCC=...; __Secure-1PSIDCC=...; VISITOR_INFO1_LIVE=...; YSC=...; _ga=...; _gid=...; _ga_HTXKR35SN9=...; CONSISTENCY=...

[Tyrrrz]

@xBaank it works now! I can pull the private video, after having deleted the other brand account and after your latest changes. I'm good with merging this as it is. Thank you for your work, it's a very useful feature :)