search

flashdagger / ytdlp-plugins

plugin manager for yt-dlp which enables releases of extractors as separate python package
The Unlicense
44 stars 4 forks source link

auf 1 broke again #6

Closed BergbauerSepp closed 1 year ago

BergbauerSepp commented 1 year ago

xxx@xxx:~/bin $ ytdlp-plugins -F -v https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-15-dezember-2022 [debug] Command-line config: ['-F', '-v', 'https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-15-dezember-2022'] [debug] Loaded 6 plugins which are not part of yt-dlp. Use at your own risk. [debug] [auf1:radio] via 'ytdlp_plugins.extractor.auf1.Auf1RadioIE' (v2022.11.28) [debug] [auf1] via 'ytdlp_plugins.extractor.auf1.Auf1IE' (v2022.11.28) [debug] [pm-wissen:search] via 'ytdlp_plugins.extractor.servustv.PmWissenSearchIE' (v2022.11.15) [debug] [pm-wissen] via 'ytdlp_plugins.extractor.servustv.PmWissenIE' (v2022.11.15) [debug] [servustv:search] via 'ytdlp_plugins.extractor.servustv.ServusSearchIE' (v2022.11.15) [debug] [servustv] via 'ytdlp_plugins.extractor.servustv.ServusTVIE' (v2022.11.15) [debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version 2022.11.11 [8b64402] (pip) [debug] Python 3.9.2 (CPython x86_64 64bit) - Linux-5.18.0-0.bpo.1-amd64-x86_64-with-glibc2.31 (OpenSSL 1.1.1n 15 Mar 2022, glibc 2.31) [debug] exe versions: ffmpeg 4.3.5-0, ffprobe 4.3.5-0, phantomjs ., rtmpdump 2.4 [debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2020.06.20, mutagen-1.45.1, pyxattr-0.7.2, sqlite3-2.6.0, websockets-10.2 [debug] Proxy map: {} [debug] Loaded 1729 extractors [debug] [auf1] Extracting URL: https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-15-dezember-2022 [auf1] nachrichten-auf1-vom-15-dezember-2022: Downloading JSON metadata WARNING: [auf1] nachrichten-auf1-vom-15-dezember-2022: Unsupported URL: https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-15-dezember-2022 [auf1] nachrichten-auf1-vom-15-dezember-2022: Downloading webpage ERROR: [auf1] nachrichten-auf1-vom-15-dezember-2022: Unable to extract payload url File "/usr/local/lib/python3.9/dist-packages/yt_dlp/extractor/common.py", line 674, in extract ie_result = self._real_extract(url) File "/usr/local/lib/python3.9/dist-packages/ytdlp_plugins/extractor/auf1.py", line 287, in _real_extract metadata = self._metadata(url, page_id=page_id, method="payloadjs") File "/usr/local/lib/python3.9/dist-packages/ytdlp_plugins/extractor/auf1.py", line 275, in _metadata payload = self._payloadjs(url, page_id) File "/usr/local/lib/python3.9/dist-packages/ytdlp_plugins/extractor/auf1.py", line 240, in _payloadjs payloadjs_url = self._search_regex( File "/usr/local/lib/python3.9/dist-packages/yt_dlp/extractor/common.py", line 1228, in _search_regex raise RegexNotFoundError('Unable to extract %s' % _name)

flashdagger commented 1 year ago 
xxx@xxx:~/bin $ ytdlp-plugins -F -v https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-15-dezember-2022
[debug] Command-line config: ['-F', '-v', 'https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-15-dezember-2022']
[debug] Loaded 6 plugins which are not part of yt-dlp. Use at your own risk.
[debug] [auf1:radio] via 'ytdlp_plugins.extractor.auf1.Auf1RadioIE' (v2022.11.28)
[debug] [auf1] via 'ytdlp_plugins.extractor.auf1.Auf1IE' (v2022.11.28)
[debug] [pm-wissen:search] via 'ytdlp_plugins.extractor.servustv.PmWissenSearchIE' (v2022.11.15)
[debug] [pm-wissen] via 'ytdlp_plugins.extractor.servustv.PmWissenIE' (v2022.11.15)
[debug] [servustv:search] via 'ytdlp_plugins.extractor.servustv.ServusSearchIE' (v2022.11.15)
[debug] [servustv] via 'ytdlp_plugins.extractor.servustv.ServusTVIE' (v2022.11.15)
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (pip)
[debug] Python 3.9.2 (CPython x86_64 64bit) - Linux-5.18.0-0.bpo.1-amd64-x86_64-with-glibc2.31 (OpenSSL 1.1.1n 15 Mar 2022, glibc 2.31)
[debug] exe versions: ffmpeg 4.3.5-0, ffprobe 4.3.5-0, phantomjs ., rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2020.06.20, mutagen-1.45.1, pyxattr-0.7.2, sqlite3-2.6.0, websockets-10.2
[debug] Proxy map: {}
[debug] Loaded 1729 extractors
[debug] [auf1] Extracting URL: https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-15-dezember-2022
[auf1] nachrichten-auf1-vom-15-dezember-2022: Downloading JSON metadata
WARNING: [auf1] nachrichten-auf1-vom-15-dezember-2022: Unsupported URL: https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-15-dezember-2022
[auf1] nachrichten-auf1-vom-15-dezember-2022: Downloading webpage
ERROR: [auf1] nachrichten-auf1-vom-15-dezember-2022: Unable to extract payload url
  File "/usr/local/lib/python3.9/dist-packages/yt_dlp/extractor/common.py", line 674,
    in extract ie_result = self._real_extract(url)
  File "/usr/local/lib/python3.9/dist-packages/ytdlp_plugins/extractor/auf1.py", line 287,
    in _real_extract metadata = self._metadata(url, page_id=page_id, method="payloadjs")
  File "/usr/local/lib/python3.9/dist-packages/ytdlp_plugins/extractor/auf1.py", line 275,
    in _metadata payload = self._payloadjs(url, page_id)
  File "/usr/local/lib/python3.9/dist-packages/ytdlp_plugins/extractor/auf1.py", line 240,
    in _payloadjs payloadjs_url = self._search_regex(
  File "/usr/local/lib/python3.9/dist-packages/yt_dlp/extractor/common.py", line 1228,
    in _search_regex raise RegexNotFoundError('Unable to extract %s' % _name)

This is so much more readable. I confirm and will look into it.

flashdagger commented 1 year ago

HTML/JS was changed. Please try the new release.

flashdagger commented 1 year ago

closing due to inactivity. feel free to reopen at any time.

BergbauerSepp commented 1 year ago

Thank you for solving the problem so quickly and sorry for my late reply.

With the new release it is again possible to download videos newer than 12.12.2022. However, downloading the playlist "https://auf1.tv/videos" is not possible for videos newer 12.12.2022. The download always starts on 12.12.2022.

xxx@xxx:~/Downloads $ date So 25 Dez 2022 21:13:27 CET xxx@xxx:~/Downloads $ python3 -m ytdlp_plugins -v https://auf1.tv/videos [debug] Command-line config: ['-v', 'https://auf1.tv/videos'] [debug] ytdlp-plugins (v2022.12.17) loaded 6 plugins which are not part of yt-dlp. Use at your own risk. [debug] [auf1:radio] via 'ytdlp_plugins.extractor.auf1.Auf1RadioIE' (v2022.12.16) [debug] [auf1] via 'ytdlp_plugins.extractor.auf1.Auf1IE' (v2022.12.16) [debug] [pm-wissen:search] via 'ytdlp_plugins.extractor.servustv.PmWissenSearchIE' (v2022.11.15) [debug] [pm-wissen] via 'ytdlp_plugins.extractor.servustv.PmWissenIE' (v2022.11.15) [debug] [servustv:search] via 'ytdlp_plugins.extractor.servustv.ServusSearchIE' (v2022.11.15) [debug] [servustv] via 'ytdlp_plugins.extractor.servustv.ServusTVIE' (v2022.11.15) [debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version 2022.11.11 [8b64402] (pip) [debug] Python 3.9.2 (CPython x86_64 64bit) - Linux-5.18.0-0.bpo.1-amd64-x86_64-with-glibc2.31 (OpenSSL 1.1.1n 15 Mar 2022, glibc 2.31) [debug] exe versions: ffmpeg 4.3.5-0, ffprobe 4.3.5-0, phantomjs ., rtmpdump 2.4 [debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2020.06.20, mutagen-1.45.1, pyxattr-0.7.2, sqlite3-2.6.0, websockets-10.2 [debug] Proxy map: {} [debug] Loaded 1729 extractors [debug] [auf1] Extracting URL: https://auf1.tv/videos [auf1] all_videos: Downloading JSON metadata [download] Downloading playlist: AUF1.TV - Alle Videos [auf1] Playlist AUF1.TV - Alle Videos: Downloading 1524 videos of 1524 [download] Downloading video 1 of 1524 [debug] [auf1] Extracting URL: https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-12-dezember-2022/ [auf1] nachrichten-auf1-vom-12-dezember-2022: Downloading JSON metadata [PeerTube] d44GkGUx1CTyCSiYGKhkAw: Downloading video JSON [debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id [PeerTube] d44GkGUx1CTyCSiYGKhkAw: Downloading description JSON [debug] Default format spec: bestvideo*+bestaudio/best [info] d44GkGUx1CTyCSiYGKhkAw: Downloading 1 format(s): 1080p-1 [debug] Invoking http downloader on "https://gegenstimme.tv/static/webseed/818116bf-5901-4414-a25e-01d1e43114af-1080.mp4" [download] Resuming download at byte 25088268 [download] Destination: Nachrichten AUF1 vom 12. Dezember 2022 [d44GkGUx1CTyCSiYGKhkAw].mp4 [download] 2.1% of 1.18GiB at 2.36MiB/s ETA 08:22^C ERROR: Interrupted by user

flashdagger commented 1 year ago

Seems like the provided a temporary JSON backend during rolling out changes. If you can please apply the following patch and check if there are other issues:

--- old/ytdlp_plugins/extractor/auf1.py
+++ new/ytdlp_plugins/extractor/auf1.py
@@ -176,7 +176,7 @@

     def call_api(self, endpoint, video_id=None, fatal=True):
         return self._download_json(
-            f"https://admin.auf1.tv/api/{endpoint}",
+            f"https://auf1.tv/api/{endpoint}",
             video_id=video_id,
             fatal=fatal,
             errnote="JSON API",
BergbauerSepp commented 1 year ago

It works :-) Big thanks!!!

It is possible to download the complete playlist as well as individual videos before and after 12.12.2022

xxxxx@xxxxx:~/Downloads $ python3 -m ytdlp_plugins -v https://auf1.tv/videos [debug] Command-line config: ['-v', 'https://auf1.tv/videos'] [debug] ytdlp-plugins (v2022.12.17) loaded 6 plugins which are not part of yt-dlp. Use at your own risk. [debug] [auf1:radio] via 'ytdlp_plugins.extractor.auf1.Auf1RadioIE' (v2022.12.16) [debug] [auf1] via 'ytdlp_plugins.extractor.auf1.Auf1IE' (v2022.12.16) [debug] [pm-wissen:search] via 'ytdlp_plugins.extractor.servustv.PmWissenSearchIE' (v2022.11.15) [debug] [pm-wissen] via 'ytdlp_plugins.extractor.servustv.PmWissenIE' (v2022.11.15) [debug] [servustv:search] via 'ytdlp_plugins.extractor.servustv.ServusSearchIE' (v2022.11.15) [debug] [servustv] via 'ytdlp_plugins.extractor.servustv.ServusTVIE' (v2022.11.15) [debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version 2022.11.11 [8b64402] (pip) [debug] Python 3.9.2 (CPython x86_64 64bit) - Linux-5.18.0-0.bpo.1-amd64-x86_64-with-glibc2.31 (OpenSSL 1.1.1n 15 Mar 2022, glibc 2.31) [debug] exe versions: ffmpeg 4.3.5-0, ffprobe 4.3.5-0, phantomjs ., rtmpdump 2.4 [debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2020.06.20, mutagen-1.45.1, pyxattr-0.7.2, sqlite3-2.6.0, websockets-10.2 [debug] Proxy map: {} [debug] Loaded 1729 extractors [debug] [auf1] Extracting URL: https://auf1.tv/videos [auf1] all_videos: Downloading JSON metadata [download] Downloading playlist: AUF1.TV - Alle Videos [auf1] Playlist AUF1.TV - Alle Videos: Downloading 1563 videos of 1563 [download] Downloading video 1 of 1563 [debug] [auf1] Extracting URL: https://auf1.tv/nachrichten-auf1/nachrichten-auf1-vom-23-dezember-2022/ [auf1] nachrichten-auf1-vom-23-dezember-2022: Downloading JSON metadata [PeerTube] rgbBUXuWNFPLJZLatxyZnU: Downloading video JSON [debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id [PeerTube] rgbBUXuWNFPLJZLatxyZnU: Downloading description JSON [debug] Default format spec: bestvideo*+bestaudio/best [info] rgbBUXuWNFPLJZLatxyZnU: Downloading 1 format(s): 1080p-1 [debug] Invoking http downloader on "https://gegenstimme.tv/static/webseed/d76afd57-ecce-461f-8bac-a3c136eaf0d5-1080.mp4" [download] Resuming download at byte 14797447 [download] Destination: Nachrichten AUF1 vom 23. Dezember 2022 [rgbBUXuWNFPLJZLatxyZnU].mp4 [download] 1.6% of 1.35GiB at 1.76MiB/s ETA 12:49^C ERROR: Interrupted by user

Edit: no other issues found