Can't download or extract_info Youtube videos from Google Cloud run service

[MonsieurSquiky]

Hi there, first of all thanks for your work, it does wonder on localhost ! But when I deploy the server on a Google Cloud Run service, I get the following log errors

2023-09-09 12:36:06.236 WET GET200648 B2.2 sPostmanRuntime/7.29.0 https://youtubedl-api-server-faych5eloa-ew.a.run.app/extract_info?url=https://www.youtube.com/watch?v=lIJGCGcA7AE 2023-09-09 12:36:07.058 WET WARNING: [youtube] unable to extract initial player response; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 12:36:07.154 WET WARNING: [youtube] Skipping player response from ios client (got player response for video "M5t4UHllkUM" instead of "lIJGCGcA7AE"); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 12:36:07.262 WET WARNING: [youtube] Skipping player response from android client (got player response for video "M5t4UHllkUM" instead of "lIJGCGcA7AE"); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 12:36:07.854 WET WARNING: [youtube] Skipping player response from web client (got player response for video "M5t4UHllkUM" instead of "lIJGCGcA7AE"); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 12:36:07.866 WET WARNING: [youtube] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 12:36:07.866 WET WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API. 2023-09-09 12:36:08.467 WET WARNING: Extractor failed to obtain "title". Creating a generic title instead 2023-09-09 12:36:08.467 WET ERROR: [youtube] lIJGCGcA7AE: No video formats found!; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on

However, it still works fine for Dailymotion videos ! I was wondering if setting cookies could solve the issue, but I can't figure out how to push a cookies file in the body for download request

If you have a hacky way to set cookies even in hardcode I'm all ears ;)

[Totonyus]

Hi !

According to the logs it gets informations form the "video not available" page : https://www.youtube.com/watch?v=M5t4UHllkUM

I don't think cookies will help a lot is this case. The video seems not geographically restricted but it's still possible the uploader restricts the video in some countries. I don't know where the google server you use are located but it could be a response.

Can you try with a different video ?

You also can try to connect directly in the container to launch directly the command to see the behavior.

And to answer : there is currently no way to provide cookies directly in request body. It's a feature I could implement with a little help (I usualy don't need cookies for my usage and I don't really know how it works for yt-dlp)

[MonsieurSquiky]

Unfortunately it does the same with any video, here's when I tried with the one you ping in the docs https://www.youtube.com/watch?v=9Lgc3TxqgHA

2023-09-09 22:35:02.271 WET [09-09-23 21:35:02][download_manager][INFO] Init download - user_token: None - presets: None - url :https://www.youtube.com/watch?v=9Lgc3TxqgHA 2023-09-09 22:35:04.283 WET [09-09-23 21:35:04][youtube-dlp][WARNING] [youtube] unable to extract initial player response; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 22:35:04.462 WET [09-09-23 21:35:04][youtube-dlp][WARNING] [youtube] Skipping player response from ios client (got player response for video "M5t4UHllkUM" instead of "9Lgc3TxqgHA"); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 22:35:04.562 WET [09-09-23 21:35:04][youtube-dlp][WARNING] [youtube] Skipping player response from android client (got player response for video "M5t4UHllkUM" instead of "9Lgc3TxqgHA"); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 22:35:05.265 WET [09-09-23 21:35:05][youtube-dlp][WARNING] [youtube] Skipping player response from web client (got player response for video "M5t4UHllkUM" instead of "9Lgc3TxqgHA"); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 22:35:05.472 WET [09-09-23 21:35:05][youtube-dlp][WARNING] [youtube] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 22:35:05.473 WET [09-09-23 21:35:05][youtube-dlp][WARNING] [youtube] Incomplete data received in embedded initial data; re-fetching using API. 2023-09-09 22:35:06.070 WET [09-09-23 21:35:06][youtube-dlp][WARNING] Extractor failed to obtain "title". Creating a generic title instead 2023-09-09 22:35:06.071 WET [09-09-23 21:35:06][youtube-dlp][ERROR] ERROR: [youtube] 9Lgc3TxqgHA: No video formats found!; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U 2023-09-09 22:35:06.073 WET [09-09-23 21:35:06][download_manager][ERROR] Checking url with preset DEFAULT => check failed 2023-09-09 22:35:06.075 WET [09-09-23 21:35:06][download_manager][ERROR] Not downloadable with presets : None : https://www.youtube.com/watch?v=9Lgc3TxqgHA 2023-09-09 22:35:06.076 WET [09-09-23 21:35:06][download_manager][ERROR] Not downloadable with presets : None : https://www.youtube.com/watch?v=9Lgc3TxqgHA 2023-09-09 22:35:06.078 WET [09-09-23 21:35:06][uvicorn.access][INFO] 169.254.1.1:16110 - "GET /download?url=https://www.youtube.com/watch?v=9Lgc3TxqgHA HTTP/1.1" 400

But this part seems interesting : got player response for video "M5t4UHllkUM" instead of "9Lgc3TxqgHA" where M5t4UHllkUM is the non available video but no idea where it comes from

I'm not sure I can access directly the container with Cloud run services, maybe I should try another kind of server : did you manage to deploy on a Heroku or any other cloud provider ?

And for the cookies feature : I would be willing to help, although I'm not that skilled... Can you point me where you the ytdl lib in your code so I can try some things ?

[Totonyus]

Hi.

I only use it in my own home server so I never tried to deploy via a cloud provider.

I'll do the coding part but the help I need is how to download using cookies. I never needed that so I'm not really sure to be able to make it work (I'll try to find out myself).

[Totonyus]

I figured how to test and I implemented it on the preview docker image.

See documentation here : https://github.com/Totonyus/ydl_api_ng/tree/develop#post-request

[MonsieurSquiky]

Amazing, I think it worked ! I was even able to get result from the GET /extract_info?url= endpoint now, without setting cookies again : do you save cookies somewhere for that ?

I'm going to keep testing but it looks good to me ! Also, would it be too much to ask to add a POST /extract_info endpoint where we can send cookies in the body same way than POST /download ?

Tell me when you want me to close the issue, and thanks a lot man ✨

[Totonyus]

It's weird. I don't know if yt-dlp has a caching system but I didn't change enything to the extract_info route.

The cookies are stored in a file under /app/cookies directory and are deleted after download or after a failed simulation. And of course on container stop.

I'll see to add a post route to the extract_info and maybe at some other endpoints if makes sense.

I'll close this issue when I'll merge the develop branch on the main branch.

Thank you for your contribution :)

[Totonyus]

POST method added for the extract info endpoint.

Merged in main branch. Containers updated.