Closed Floflo10 closed 7 months ago
Ugh, I'm very sorry that you came across that error, in my experience the only solution that lasts over time is on my main desktop to run an instance of Firefox every 20 minutes in the background, playing any chapter and closing the instance after 60 seconds, With that I configure crunchyroll_auth in "browser", crunchyroll_browser (with the firefox path: profile-path) and in firefox user agent. With the use of auth through cookies, every 20 minutes it requires a cookie refresh. If you are in an environment without x, you can get it with https://github.com/SeleniumHQ/docker-selenium or https://opensource.zalando.com/zalenium/ This should fix the problems with cloudflare.
First, thank a lot for your answer !
I struggle a lot but I think I success in a inelegant and heavy way using chrome in a neko container. I think I can do something more clean and automated with the docker image you recommand. I will try and post it if I can find a way.
However, I hit another wall maybe linked to the initial cookie expiration. I got now a constant 403 error
"Request blocked. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner. If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation. "
I took a look at the repo of yt-dlp and it seems like the API changed recently https://github.com/yt-dlp/yt-dlp/issues/9439 https://github.com/yt-dlp/yt-dlp/issues/9453
I think the script will not work anytime soon :/
Hello! I've found myself in a similar situation as @Floflo10 and checking logs I've found the following:
Jellyfinn is reporting that ytdlp2STRM is returning a 403
http://ytdlp2STRM:5000/crunchyroll/direct/es_watch_G63K39W46: Server returned 403 Forbidden (access denied)
Nonetheless, via curl it returns a 302 as expected with the crunchyroll URL:
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to the target URL: <a href="https://pl.crunchyroll.com/evs3/dcc84f46ba09918193fda12ca6ff168e/assets/8fa35386681f1e16a6e68ce999269508
_,5290156.mp4,5290157.mp4,5290155.mp4,5290153.mp4,5290154.mp4,.urlset/master.m3u8? etc... etc...
If not, click the link.
But after that redirect, we are received by a 403 from cloudflare that @Floflo10 indicated
curl https://pl.crunchyroll.com/evs3/dcc84f46ba09918193fda12ca6ff168e/assets/8fa35386681f1e16a6e68ce999269508_,5290156.mp4,5290157.mp4,5290155.mp4,5290153.mp4,5290154.mp4,.urlset/master.m3u8\?Policy\=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cCo6Ly9wbC5jcnVuY2h5cm9sbC5jb20vZXZzMy9kY2M4NGY0NmJhMDk5MTgxOTNmZGExMmNhNmZmMTY4ZS9hc3NldHMvOGZhMzUzODY2ODFmMWUxNmE2ZTY4Y2U5OTkyNjk1MDhfLDUyOTAxNTYubXA0LDUyOTAxNTcubXA0LDUyOTAxNTUubXA0LDUyOTAxNTMubXA0LDUyOTAxNTQubXA0LC51cmxzZXQvbWFzdGVyLm0zdTgiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3MTE3MzA5ODZ9fX1dfQ__\&Signature\=MEzuVQaccjNbPOWDIKoMQfUezJ5azgx-Tpd2rg92X0xCe2CqgeZAFIUOXZ2bnW714VpELtcVM\~g9MMZ7Iy3FZBZcpAn-c6yGc2NQXo96t9dTIMOwZIUPyfk4cKm3ODt2ERCN0UACiX4ZyUotl8YCN1FBR4JjkAs44dF1X7i0OlBROEtPu\~koLrhg-9quhV98emFNSc34r9ycNijJ7L3GUstHQw5SlXVtyBQDPa-NleeEmthUEy95\~xjTDlk1XDZFdGnfp-xbnpkvrxYOv-9m5vqJ495dHciXZU5Gy6gl0R2h21udp-l7jDbO3kR6pKQ2lw3U5idIVx0MI9wxNbxq8w__\&Key-Pair-Id\=APKAJMWSQ5S7ZB3MF5VA
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>403 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Request blocked.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: LpOQhS-XEwDl5uCpf1EJkF6SXojUy86FFS9Ji_Or92PQVBulFYxeoA==
Edit: Found what they report as a workaround, haven't tested it yet
https://github.com/yt-dlp/yt-dlp/issues/9439#issuecomment-1997864883
--extractor-args "crunchyrollbeta:format=vo_adaptive_hls"
@DarkYamik Yeah, I tested the workaround before my last comment and didn't had success. Have you tested it since your post ?
It seem like a potential fix was merged : https://github.com/yt-dlp/yt-dlp/pull/9615 What do you think all ?
That PR doesn’t fix the Cloudflare error
the solution is to pass --impersonate=“”
in the yt-dlp command. That option will be enabled by default soon
That PR doesn’t fix the Cloudflare error
the solution is to pass
--impersonate=“”
in the yt-dlp command. That option will be enabled by default soon
Hi, the downloads stop with this. (My command line has error?) But cookie work fine and stop every ≈ 30min.
[debug] Command-line config: ['-P', 'PATH', '--username', 'PRIVATE', '--password', 'PRIVATE', '--impersonate=', '-f', 'bv+ba/b', '--sub-langs', 'fr-FR,fr', '--embed-subs', '--remux-video', 'mkv', '-Uv', '-a', 'PATH']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2024.04.06.232655 from yt-dlp/yt-dlp-nightly-builds [b15b0c1d2] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.22621-SP0 (OpenSSL 1.1.1k 25 Mar 2021)
[debug] exe versions: ffmpeg 6.0-full_build-www.gyan.dev (setts), ffprobe 6.0-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.31.0, sqlite3-3.35.5, urllib3-2.2.1, websockets-12.0
My command line has error?
Did you remove the command line config
line? I can't help you debug command line errors if you don't include it, but feel free to redact it
EDIT: also, what do you mean by "the downloads stop"? Do they run for a few minutes and then crash? What error?
My command line has error?
Did you remove the
command line config
line? I can't help you debug command line errors if you don't include it, but feel free to redact itEDIT: also, what do you mean by "the downloads stop"? Do they run for a few minutes and then crash? What error?
[debug] Command-line config: ['-P', 'E:\\YTdlp\\Crunchyroll', '--username', 'PRIVATE', '--password', 'PRIVATE', '--impersonate=', '-f', 'bv+ba/b', '--sub-langs', 'fr-FR,fr', '--embed-subs', '--remux-video', 'mkv', '-Uv', '-a', 'C:\\Users\\micka\\Downloads\\Google Drive\\Privé\\Plex\\dl_cr.txt']
Yt-dlp just stop.
[crunchyroll] Extracting URL: https://www.crunchyroll.com/fr/watch/G14U47QZG/return-to-the-land-of-the-dead
[crunchyroll] G14U47QZG: Downloading object info JSON (cms)
[crunchyroll] G14U47QZG: Downloading stream info
[crunchyroll] Available hardsub languages: en-US, de-DE, es-419, es-ES, fr-FR, it-IT, ru-RU
[crunchyroll] G14U47QZG: Downloading MPD manifest
[crunchyroll] G14U47QZG: Downloading chapter info
[info] G14U47QZG: Downloading subtitles: fr-FR
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[info] G14U47QZG: Downloading 1 format(s): f2-v1-x3+f1-a1-x3
[info] Writing video subtitles to: E:\YTdlp\Crunchyroll\SAINT SEIYA: Knights of the Zodiac Battle of Sanctuary Part II (voix anglaises) Episode 3 – Retour au royaume des morts [G14U47QZG].fr-FR.ass
[debug] Invoking http downloader on "https://v.vrv.co/evs3/c4183ac197ec493181334ca6155fa2e8/assets/2b8856cc7fc5a22442952f58e9e61ae0_383421.txt?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly92LnZydi5jby9ldnMzL2M0MTgzYWMxOTdlYzQ5MzE4MTMzNGNhNjE1NWZhMmU4L2Fzc2V0cy8yYjg4NTZjYzdmYzVhMjI0NDI5NTJmNThlOWU2MWFlMF8zODM0MjEudHh0IiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzEyNTg5MDk4fX19XX0_&Key-Pair-Id=APKAJMWSQ5S7ZB3MF5VA&Signature=Fh67plzYbJWC7XPwCtvyKF1tU7o~a49suYEBxQGclEUPBYBedHf~TLVpc~dypA2tVPIrl2LCrqTlWMRg8TZDGLUJKoBPi5nezT35oUgZcBeRgqD0bq5VvmXwdWZ0OxV53Kw6kdOJdDVae-43tqpA52U-Dc~adfIrr9MMXVMnHEuUWlBYykNPkFpl5w0VU37frNb14NHUvo3q3~khoZC5F0ZVP6UBo7iTj2uJssRFRCk3oHc6YJ1BinBNI5t~xTohWQxHnuJ1O90uSQL5pLBG2dnhVMnAF99uEbYgplWjl6BSewr8-wheID3CoH0Rmapxw7SNTYvJifvurEhrdDb7IQ__"
[download] Destination: E:\YTdlp\Crunchyroll\SAINT SEIYA: Knights of the Zodiac Battle of Sanctuary Part II (voix anglaises) Episode 3 – Retour au royaume des morts [G14U47QZG].fr-FR.ass
[download] 100% of 33.84KiB in 00:00:00 at 134.11KiB/s
[debug] Invoking dashsegments downloader on "https://cr-play-service.prd.crunchyrollsvc.com/v1/manifest/G14U47QZG/evs3/c4183ac197ec493181334ca6155fa2e8/assets/p/2b8856cc7fc5a22442952f58e9e61ae0_,5502074.mp4,5502075.mp4,5502073.mp4,5502071.mp4,5502072.mp4,.urlset/manifest.mpd?playbackGuid=08-632157c6-8840-47d6-9ba6-dc7d39b2e4fa&accountid=b46f76ad-a6b3-5531-bfd3-573afb953122"
[dashsegments] Total fragments: 346
[download] Destination: E:\YTdlp\Crunchyroll\SAINT SEIYA: Knights of the Zodiac Battle of Sanctuary Part II (voix anglaises) Episode 3 – Retour au royaume des morts [G14U47QZG].ff2-v1-x3.mp4
[download] 25.6% of ~ 1.25GiB at 7.56MiB/s ETA 02:20 (frag 89/346)
C:\Users\micka>
Thx for your help and fast response.
I added download mode for Crunchyroll. It's the only way to continue working at Jellyfin. Direct mode runs download mode to avoid editing all your strm files.
A very serious drawback is that you have to wait for the file to download and remux before serving it.
*With 2 threads and post-processing of mp4 (video) and m4a (audio) files and 1 Gbps connection takes approximately 45 seconds to load the video on Jellyfin.
New feature!! Check last commit https://github.com/fe80Grau/ytdlp2STRM/commit/4fa5ba133014d9ad372bd54584284ef9861343df
I added plugins/crunchyroll/jellyfin.py and 4 more variables in plugins/crunchyroll/config.json
This feature queries the Jellyfin API for the given user_id to get if there are any Crunchyroll items playing, if so it will locate the next episode and if there is it will launch the requests for ytdlp2STRM to download and build the new episode in the temp folder, like this While viewing an episode, the next one is downloaded and viewing is instantaneous.
Wow, that's neat ! Thanks a lot for your fast response and the new feature. I'll be testing it over the weekend.
Just to make sure I understand, but now :
Is that right? :)
It's something like that, but let me clarify one point. The functionality starts working when you start a specific episode for the first time in Jellyfin (in the last 24 hours), then you will have to wait for it to download between 1 or 3 minutes, but while it is playing ytdlp2STRM will detect it and search if it exists a next episode, if so, start downloading it.
The idea with ytdlp2STRM is that storage is not consumed unnecessarily, which is why it does not start downloading until there is actually a demand for the episode.
In any case, the files will always be served from temp/
Files in the temp/ folder older than 24 hours will be automatically deleted.
I don't know if I make myself understood.
P.S: Update to latest commit, I forgot to add the temp folder.
P.S 2: You gave me the idea that the last chapter detected is good that it is automatically downloaded in temp to access quickly the same day it comes out... I will add it as an option
What is promised is a debt. Check last commit.
I added a new config option in Jellyfin plugin. jellyfin_preload_last_episode, set True to preloads the last episode at the time its strm is generated. Remember in 24h will be deleted from temp folder
First of all : you're amazing ! (The new cli is superb btw)
It doesn't seems to work but it's close. I got this error when I tried to launch a video from Jellyfin :
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/opt/ytdlp2STRM/plugins/crunchyroll/routes.py", line 7, in crunchyroll_direct
return direct(crunchyroll_id)
File "/opt/ytdlp2STRM/plugins/crunchyroll/crunchyroll.py", line 320, in direct
return download(crunchyroll_id)
File "/opt/ytdlp2STRM/plugins/crunchyroll/crunchyroll.py", line 384, in download
preprocess_video(
File "/opt/ytdlp2STRM/plugins/crunchyroll/crunchyroll.py", line 344, in preprocess_video
subprocess.run(cmd, check=True)
File "/usr/local/lib/python3.9/subprocess.py", line 505, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/local/lib/python3.9/subprocess.py", line 951, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/lib/python3.9/subprocess.py", line 1837, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg'
172.23.0.1 - - [14/Apr/2024 22:26:46] "GET /crunchyroll/direct/fr_watch_G31UX4N7G HTTP/1.1" 500 -
Maybe a bit more context if if that help. I use Docker / docker compose for my service. Because ytdlp2STRM :latest don't deliver your last commit I build myself the image from the repo (and in requierments.txt tzlocal
was lacking). Maybe it's come from the docker env ?
(Oh, before I forgot, the cli doesn't seem to display the error message when cloudflare say than the cookies have expired).
Suggestion : perhaps, if possible, let the use decide how long the files in temporary folder can last before being deleted. Some people have a bad combo low time + bad connection. So they pre-download thieir content for a later view. With a little control over this aspect, storage is always used as it should be !
In ant case, thanks a LOT !
From the error output it seems that it does not recognize ffmpeg, but it is strange because I include it in the requirements.txt...
Anyway I've added tzlocal as part of the requirements.txt, I've also pushed the latest mods to docker hub so ydlp2STRM:latest should build the image correctly once
I have also added the parameter ytdlp2strm_temp_file_duration in ./config/config.json by default at 86400 (it is in seconds and is equivalent to 24h) simply modify it to the value you want (in seconds) and restart the service.
EDIT: I just added a command to correctly update and install ffmpeg in Docker and I have updated requierments for it to install ffmpeg-python, I think this way should fix the above error.
I hope it all works out
I tried several combinations and everything worked perfectly, it's perfect ! I think we can close the issue. I will try to tinker something on my side with docker container for a auto refresh cookies. I will open a new issue if I can achieve something :D
Thanks a lot again !
@Floflo10 pull the latest docker image, I fixed a bug with the function that cleans the temp folder and with the previous image it is never deleted
@fe80Grau Thanks a lot ! I pulled it, I will check with thhe new bach of the week if it work.
I don't know if it's linked but I got a 403 error when I try to use it :
Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: Forbidden>)
Oh boy not again I'm cursed ! x) It seems it's tracked here : https://github.com/yt-dlp/yt-dlp/issues/9745 I guess we just have to wait 🙃
yes time to wait... it's a pitty
I'm trying modifying the crunchyroll extractor code for yt-dlp and looking at the code at https://github.com/crunchy-labs/crunchy-cli, but I don't think I can find the key without a Switch with CFW
If yt-dlp takes a long time to solve, I will try to implement crunchy-cli for ytdlp2STRM, the main problem I see is the integration, I integrate yt-dlp easily with pip, but crunchy-cli is written in rust and the management of its binaries depends on different package managers for different OS, nothing impossible but I sincerely hope that yt-dlp is fixed soon.
@Floflo10 is working again with https://github.com/yt-dlp/yt-dlp/pull/9749/commits/cb62cecda4ea58de347f10c1fb2cd567af82f1d4 changes. I think it should be fixed in today's nightly build.
I will update requirements.txt with today's nightly version once it comes out
@fe80Grau Wow, that was fast ! There are more twist than a netflix serie, I will test it asap. Thanks a lot for your hard work :) If I understand correctly, cookies are no longer needed?
requierments updated and another fix for temp files https://github.com/fe80Grau/ytdlp2STRM/commit/a22839107fd6e802c9440a8151cb18257c0d285c
I pushed to latest docker image too
Thanks a lot ! I just tested it and it's look like download failed at some point :
ERROR: Unable to download video: [Errno 2] No such file or directory: '/opt/ytdlp2STRM/temp/fr_watch_GJWU200VZ.mp4.part-Frag29'
/opt/ytdlp2STRM/temp/fr_watch_GJWU200VZ.mp4: No such file or directory
[2024-04-23 22:09:42,015] ERROR in app: Exception on /crunchyroll/direct/fr_watch_GJWU200VZ [GET]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/opt/ytdlp2STRM/plugins/crunchyroll/routes.py", line 7, in crunchyroll_direct
return direct(crunchyroll_id)
File "/opt/ytdlp2STRM/plugins/crunchyroll/crunchyroll.py", line 322, in direct
return download(crunchyroll_id)
File "/opt/ytdlp2STRM/plugins/crunchyroll/crunchyroll.py", line 387, in download
preprocess_video(
File "/opt/ytdlp2STRM/plugins/crunchyroll/crunchyroll.py", line 343, in preprocess_video
ffmpeg.output(
File "/usr/local/lib/python3.9/site-packages/ffmpeg/_run.py", line 325, in run
raise Error('ffmpeg', out, err)
ffmpeg._run.Error: ffmpeg error (see stderr output for detail)
192.168.1.203 - - [23/Apr/2024 22:09:42] "GET /crunchyroll/direct/fr_watch_GJWU200VZ HTTP/1.1" 500 -
Now, I've finally figured it out. It was hard to identify the issue because I was focusing on the function that extracts the last modified date, thinking something was wrong there. However, it turned out to be a simple parameter I needed to add to the yt-dlp command line to ensure that the downloaded files would have the current date, instead of being set according to the release date of the item.
--no-mtime Do not use the Last-modified header to set the file modification time
Now everything should work correctly, both, here and Docker, which is already updated. I hope it works!
I've tested it and it works, I don't even need to build a stack to automatically update the cookies ! I've had a few episodes with no audio after a few minutes or with images freezing but I think it's due to my old rig ^^' Thanks a lot again ! 💪 (I think we can close it again... Until next time 🙃)
Hi guys !
I think I just miss something but, after hours failling to find a fix, I'm out of idea. I installed the docker image a few week ago on a LXC in my proxmox server. I struggle a bit but, finally, everything was working wonderfully well. I could get crunchyroll on my jellyfin. But, yesterday, it stopped working and I saw in logs the classic:
Request blocked by Cloudflare; navigate to Crunchyroll in your browser, then pass the fresh cookies (with --cookies-from-browser or --cookies) and your browser's User-Agent (with --user-agent)
From here, I tried to udpate my cookies again and nothing worked. I have tried to get cookies and user-agent from 3 differents browser from my usual client and still get error. I also tried to use the "browser" authentication method with firefox in a container + bind mount and it don't work as either.
Did I miss something ? Or my use case cause all of this ?
In any case, thank a lot for your work!