ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.7k stars 9.97k forks source link

viki.com: download fails with 401 unauthorized for some shows #26246

Open kytrinyx opened 4 years ago

kytrinyx commented 4 years ago

Checklist

Verbose logs

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--write-sub', '--sub-lang', 'ko', '--sub-format', 'srt', '--skip-download', '-o', '/Users/kytrinyx/Dropbox/katrina/korean/subtitles/all/three-days/%(title)s-%(id)s.%(ext)s', 'https://www.viki.com/tv/12697c-three-days']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 3.8.5 (CPython) - macOS-10.14.6-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[viki:channel] 12697c: Downloading channel JSON
[viki:channel] 12697c: Downloading episodes JSON page #1
[viki:channel] 12697c: Downloading clips JSON page #1
[viki:channel] 12697c: Downloading movies JSON page #1
[download] Downloading playlist: Three Days
[viki:channel] playlist Three Days: Collected 16 video ids (downloading 16 of them)
[download] Downloading video 1 of 16
[viki] 1033605v: Downloading video JSON
[viki] 1033605v: Downloading video streams JSON
[debug] Default format spec: bestvideo+bestaudio/best
[info] Writing video subtitles to: /Users/kytrinyx/Dropbox/katrina/korean/subtitles/all/three-days/Three Days - Episode 1-1033605v.ko.srt
[download] Downloading video 2 of 16
[viki] 1033673v: Downloading video JSON
[viki] 1033673v: Downloading video streams JSON
[debug] Default format spec: bestvideo+bestaudio/best
[info] Writing video subtitles to: /Users/kytrinyx/Dropbox/katrina/korean/subtitles/all/three-days/Three Days - Episode 2-1033673v.ko.srt
[download] Downloading video 3 of 16
[viki] 1034112v: Downloading video JSON
[viki] 1034112v: Downloading video streams JSON
ERROR: Unable to download JSON metadata: HTTP Error 401: Unauthorized (caused by <HTTPError 401: 'Unauthorized'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--username=PRIVATE', '--password=PRIVATE', '--ignore-errors', '--write-sub', '--sub-lang', 'ko', '--sub-format', 'srt', '--skip-download', '-o', '/Users/kytrinyx/Dropbox/katrina/korean/subtitles/all/three-days/%(title)s-%(id)s.%(ext)s', 'https://www.viki.com/tv/12697c-three-days']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 3.8.5 (CPython) - macOS-10.14.6-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[viki:channel] Logging in
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/Users/kytrinyx/.pyenv/versions/3.8.5/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

Description

For a number of shows on viki.com, the API appears to return an unexpected Unauthorized 401. If I pass my login credentials, the error changes to a 404. I've pasted the verbose logs from both types of call for the show Three Days as an example.

The v4 REST API reports this show as having 92% subtitle completion in Korean. If I add --ignore-errors, then subtitles are downloaded for the 2 first episodes, and the rest error out with 401 Unauthorized (or 404 not found, if I authorize).

The metadata for the show doesn't say that it is restricted:

API v4 JSON response for Three Days ``` { "id": "12697c", "type": "series", "subtype": "normal", "content_owners": [ { "id": "352co", "resource_id": "533248" } ], "origin": { "country": "kr", "language": "ko" }, "created_at": "2013-03-28T18:07:10Z", "updated_at": "2020-07-27T04:17:35Z", "subscriptions": { "count": 25669 }, "team": { "id": "12697ct", "name": "The Three Days Team" }, "titles": { "ko": "쓰리데이즈", "hu": "Három Nap", "pt": "Three Days", "id": "Three Days", "el": "Τρεις Ημέρες", "sr": "Три дана (преведено)", "es": "Tres Días", "ro": "Trei Zile", "tr": "Üç Gün", "en": "Three Days", "it": "Three Days (2014) - Tre Giorni", "fr": "Three Days", "zt": "危情三天", "zh": "危情三天" }, "images": { "poster": { "url": "https://6.vikiplatform.com/image/402ab8944acd474db90c498a1f66d7ea.jpeg?x=b", "source": "viki" }, "atv_cover": { "url": "https://6.vikiplatform.com/image/d1ae49c822c0499cb8babfbd7f570120.jpg?x=b&a=0x0", "source": "viki" } }, "descriptions": { "tr": "Üç gün bir Ülkenin kaderini değiştirebilir. Dizi Kore Başkanlık sarayındaki gizli servis ajanları arasında üç gün süren bir olayı anlatıyor. Güney Kore Başkanı tatil için bir villaya gider. Üç silah sesinden sonra Başkan kaybolur. Koruması Han Tae Kyung ( Park Yoo Chun) Başkanı bulmaya çalışır.", "ro": "O noua productie promitatoare SBS cu lansarea pe data de 5 Martie, 2014.\n\"Presedintele tarii dispare din casa lui de vacanta in timpul concediului sau de odihna, iar garzile lui personale sunt nevoiti sa infrunte multe situatii si obstacole pentru a-l escorta inapoi la Blue House in siguranta.\"", "sr": "Три дана могу променити судбину земље. Председник Јужне Кореје, Ли Донг Хви, одлази у удаљену вилу на одмор. Три пуцња одјекују и председник нестаје. Хан Те Кјонг и његове колеге из Тајне службе Плаве куће, уз помоћ полицијског наредника Јун Бо Вон имају само три дана да истраже мистериозни нестанак и пронађу председника. Може ли Те Кјонг очекивати помоћ или ометање од људи који га окружују, укључујући и главног секретара Шин Кју Ђина, агента специјалних снага Ли Ча Јонга и начелника председничке безбедносне службе, Хан Бонг Суа? Може ли Те Кјонг победити време да нађе председника? \"Три дана\" јје јужнокорејска драма из 2014. године, у режији Шин Кјунг Суа.", "hu": "3 nap megváltoztathatja egy ország végzetét. A dél-koreai elnök, Lee Dong Hwi (Son Hyun Woo) egy távoli villába megy vakációzni. 3 pisztolylövés dördül el, és az elnök eltűnik. Han Tae Kyungnak (Park Yoochun) és kollégáinak a Kék Ház Titkos Szervezetében, Yoon Bo Won (Park Ha Sun) rendőr segítségével, 3 napjuk van, hogy kivizsgálják a rejtélyes eltűnést és, hogy megtalálják az elnököt. Várhat-e segítséget vagy akadályozást Tae Kyung a körülötte lévőktől, beleértve Shin Kyu Jin (Yoon Je Moon) főtitkárt, Lee Cha Young (So I Hyun) SWAT ügynököt és Ham Bong Soo-t (Jang Hyun Sung), az elnökbiztonsági szolgálat főnökét? Le tudja-e győzni Tae Kyung az időt, hogy megtalálja az elnököt?\nA \"Három Nap\" egy 2014-es koreai drama sorozat Shin Kyung Soo rendezőtől.", "id": "Tiga hari bisa merubah takdir sebuah negara. Presiden Korea Selatan Lee Dong Hwi (Son Hyun Joo) pergi ke sebuah villa terpencil untuk berlibur. Tiga tembakan senjata berbunyi, dan Presiden menghilang. Han Tae Kyung (Park Yoochun) dan rekannya di Agen Keamanan Rahasia Blue House, dengan bantuan seorang petugas Polisi Yoon Bo Won (Park Ha Sun), hanya mempunyai tiga hari untuk menyelidiki menghilangnya Presiden yang misterius dan menemukannya. Bisakah Tae Kyung mengharapkan bantuan atau malah gangguan dari orang-orang di sekitarnya, termasuk Kepala Sekretaris Shin Kyu Jin (Yoon Je Moon), Agen Tim SWAT Lee Cha Young (So I Hyun), dan Agen Sekuriti Kepresidenan Ham Bong Soo (Jang Hyun Sung)? Dapatkah Tae Kyung mengalahkan waktu untuk menemukan Presiden? “Three Days” adalah drama seri Korea Selatan tahun 2014 yang disutradarai oleh Shin Kyung Soo.", "fr": "Trois jours peuvent changer le destin d'un pays. Le Président Sud-Coréen Lee Dong Hwi (Son Hyun Joo) part en vacances dans une villa isolée. Trois coups de feu retentissent, et le président disparaît. Han Tae Kyung (Park Yoo Chun) et ses collègues des services secrets de la Maison Bleue, avec l'aide de l'agent de police Yoon Bo Won (Park Ha Sun), n'ont que trois jours pour enquêter sur la mystérieuse disparition du président et le trouver. Est-ce que Tae Kyung peut attendre de l'aide ou non de son entourage, en incluant le secrétaire général Shin Kyu Jin (Yoon Je Moon), l'agent du SWAT Lee Cha Young (So Yi Hyun), et le chef de la sécurité présidentielle Ham Bong Soo (Jang Hyun Sung) ? Est-ce que Tae Kyung arrivera à courir contre la montre pour trouver le président ? “Three Days” est un drama sud-coréen réalisé en 2014 et dirigé par Shin Kyung Soo.", "es": "En tres días el destino de un país puede cambiar. El Presidente Sur Coreano Lee Dong Hwi (Son Hyun Joo) va de vacaciones a un pueblo lejano. El presidente desaparece luego de escucharse tres disparos. Han Tae Kyung (Park Yoochun) y sus colegas del Servicio Secreto de la Casa Azul y con la ayuda del Oficial de Policía Yoon Bo Won (Park Ha Sun) tienen solo tres días para investigar la misteriosa desaparición del presidente y encontrarlo. ¿Puede Tae Kyung esperar ayuda o obstáculos de aquellos que lo rodean incluyendo al Jefe de Secretaría Shin Kyu Jin (Yoon Je Moon), Agente del SWAT Lee Cha Young (So I Hyun), y el Jefe del Servicio de Seguridad Presidencial Ham Bong Soo (Jang Hyun Sung)? ¿Puede Tae Kyung encontrar al presidente a tiempo? \"Tres Días\" es una serie Sur Coreana del 2014 dirigida por Shin Kyung Soo.", "el": "Τρεις ημέρες μπορούν να αλλάξουν την μοίρα ενός έθνους. Ο Νότιοκορεάτης Πρόεδρος Λι Ντονγκ Χι πάει σε μια απομακρυσμένη βίλα για διακοπές. Εκεί, ηχούν τρεις πυροβολισμοί, και ο Πρόεδρος εξαφανίζεται. Ο Χαν Τε Κιονγκ και οι συνεργάτες του από την μυστική υπήρεσία του Μπλε Οίκου, με την βοήθεια της Αστυφύλακα Γιουν Μπο Γον, έχουν μόνο τρεις ημέρες για να ερευνήσουν την μυστηριώδη εξαφάνιση και να βρουν τον Πρόεδρο.\nΜπορεί ο Τε Κιονγκ να περιμένει βοήθεια ή δυσχέρεια από αυτούς που βρίσκονται γύρω του, συμπεριλαμβανομένου και του αρχιγραμματέα Σιν Κιου Τζιν, της αρχηγού ομάδας κρούσης Λι Τσα Γιονγκ, και του αρχηγού της ομάδας προστασίας του Προέδρου Χαμ Μπονγκ Σου; Μπορεί ο Τε Κιονγκ να κερδίσει τον χρόνο που έχει και να βρει τον Πρόεδρο;", "pt": "Três dias podem mudar o destino de um país. O presidente sul-coreano Lee Dong Hwi (Son Hyun Joo) viaja para uma vila para desfrutar suas férias. Depois que três tiros são disparados, ele desaparece. Han Tae-Kyung (Park Yoochun) e seus colegas do Serviço Secreto da Casa Azul, com a ajuda da policial Yoon Bo Won (Park Ha Sun), têm somente três dias para investigar esse desaparecimento misterioso e encontrar o presidente. Poderá Tae Kyung esperar ajuda ou um obstáculo daqueles ao seu redor, incluindo o Secretário Chefe Shin Kyu Jin (Yoon Je Moon), a Agente da SWAT Lee Cha Young (So I Hyun), e o Chefe do Serviço de Segurança do Presidente Ham Bong Soo (Jang Hyun Sung)? Poderá Tae Kyung vencer o relógio e encontrar o presidente?\n\"Three Days\" é um drama sul-coreano de 2014 dirigido por Shin Kyung Soo.", "it": "Tre giorni possono cambiare il destino di un Paese. Il Presidente Sudcoreano Lee Dong Hwi (Son Hyun Joo) decide di trascorrere le sue vacanze in una villa lontana, ma scompare dopo il rimbombo di tre colpi d'arma da fuoco. Han Tae Kyung (Park Yoochun) ed i suoi colleghi dei servizi segreti della Casa Blu, con l'aiuto dell'agente di polizia Yoon Bo Won (Park Ha Sun), hanno solo tre giorni per indagare sulla misteriosa scomparsa e trovare il Presidente. Le persone attorno a Tae Kyung, tra cui il Capo Segretario Shin Kyu Jin (Yoon Je Moon), l'agente dell'Unità Speciale Lee Cha Young (So I Hyun) e Capo della Sicurezza Presidenziale Ham Bong Soo (Jang Hyun Sung), lo aiuteranno o lo ostacoleranno? Tae Kyung vincerà questa corsa contro il tempo per trovare il Presidente? \"Three Days\" è un drama sud coreano del 2014 diretto da Shin Kyung Soo.", "ko": "사선(死線)에서... 1년 365일 죽음을 받아들이는 훈련을 하는 사람들. 대한민국의 국가원수를 지키는 살아있는 방패, 청와대 대통령 경호실 경호관. 그들의 원칙은 제 목숨을 바쳐 대통령을 지키는 것. 그 대통령이 어떠한 비리나 범죄를 저지르더라도 대신 죽을만한 가치가 없는 사람이라 할지라도... 경호관들은 대통령을 위해 기꺼이 목숨을 바친다. 국민들의 투표를 통해 선출된 대통령을 지키는 것이 대의민주주의를 지키는 것이기 때문이다. 집에서 나올 때도 “다녀올게” 라는 말을 아낄 수밖에 없는... 국가안보의 최전선에 서서, 대통령을 지키는 경호관들의 숨겨진 내면을 세밀하게 그려보고자 한다. Three days 9일간의 기록... 제 1장 전쟁의 서막 제 2장 결 전 제 3장 심 판 각각의 제한된 72시간, 9일간의 기록. 그 안에 누군가는 대통령을 죽여야 하고, 누군가는 대통령을 지켜야만 한다. 시시각각 조여 오는 날선 시간 위로, 다양한 인간 군상들의 이야기를 긴박감 넘치게 펼쳐보고자 한다. 그들이 있기에 “진실은 전진 한다.” 대통령, 경호관, 정치인, 군인, 경찰, 변호사.. 누군가의 자식, 부모, 배우자, 친구... 얽히고설킨 극중의 수많은 인물들은 나쁜 사람이기도 하고 좋은 사람이기도 하며 소신과 원칙에 따라 움직이다가도 일순 흔들려 고뇌하기도 한다. 하지만 원칙과 소신을 지키며 제자리를 지키는, 진정성 있는 사람들이 있다. 쉬워 보이지만, 쉽지 않은 그들의 원칙. 그렇기 때문에 흔들리기도 하고, 더 쉬운 길을 생각해 보지만, 결국엔... 돌아올 수밖에 없는 사람들... 그들을 통해 포기하지 않은 1퍼센트가 세상을 조금이라도 바꿔나간다는 정의를 보여주고자 한다.", "en": "When the president goes missing from his vacation home without a trace, his bodyguard gets on the case and tracks him down in this intense action drama.", "zt": "總統在別墅度假時神祕失蹤且毫無線索,他的保鏢將在這部緊湊的動作連續劇中,負責調查這起案件並追蹤總統的下落。", "zh": "总统在别墅度假时神祕失踪且毫无线索,他的保鏢将在这部紧凑的动作连续剧中,负责调查这起案件并追踪总统的下落。" }, "genres": [ "1038g", "1g", "23g", "26g", "7g", "9g" ], "subtitle_completions": { "ar": 97, "bg": 0, "cs": 0, "da": 83, "de": 100, "el": 100, "en": 100, "es": 100, "fi": 99, "fr": 100, "he": 100, "hu": 94, "id": 100, "it": 100, "ja": 100, "ko": 94, "ku": 0, "lt": 0, "nl": 69, "pl": 100, "pt": 100, "ro": 100, "sh": 97, "si": 99, "sk": 100, "sq": 0, "sr": 100, "th": 100, "tl": 100, "tr": 100, "vi": 100, "zh": 100, "zt": 100 }, "managers": [ { "id": "571551u", "username": "lazarini", "images": { "avatar": { "url": null } }, "url": { "web": "https://www.viki.com/users/lazarini", "api": "https://api.viki.io/v4/users/571551u.json" } }, { "id": "291822u", "username": "serenasf95", "images": { "avatar": { "url": "https://0.viki.io/u/291822u/Jtm9EieUOy.gif" } }, "url": { "web": "https://www.viki.com/users/serenasf95", "api": "https://api.viki.io/v4/users/291822u.json" } } ], "url": { "web": "https://www.viki.com/tv/12697c-three-days", "api": "https://api.viki.io/v4/series/12697c.json", "fb": "https://www.viki.com/tv/12697c-three-days" }, "flags": { "licensed": true, "hosted": true, "state": "normal", "adult": false, "on_air": false, "exclusive": false, "original": false }, "rating": "PG-13", "review_stats": { "average_rating": 9.08, "count": 384 }, "episodes": { "count": 16, "url": { "api": "https://api.viki.io/v4/series/12697c/episodes.json" } }, "trailers": { "count": 2, "url": { "api": "https://api.viki.io/v4/series/12697c/trailers.json" } }, "titles_aka": { }, "titles_phonetic": { "en": "Sseurideizeu" }, "day_of_week": [ "wed", "thu" ], "planned_episodes": 16, "distributors": [ { "type": "broadcaster", "name": "SBS, KOCOWA", "from": "2014-03-05", "to": "2014-04-30" } ], "watch_now": { "id": "1033605v", "type": "episode", "url": { "web": "https://www.viki.com/videos/1033605v-three-days-episode-1", "api": "https://api.viki.io/v4/videos/1033605v.json" }, "episode": 1 }, "verticals": [ { "effective_from": "2019-12-12T08:00:00Z", "free": 2, "timed": 0, "vertical_id": "2pv", "url": { "api": "https://api.viki.io/v4/verticals/2pv.json", "web": "https://www.viki.com/verticals/this_is_a_stub.json" }, "type": "vertical", "released_at": "2020-08-06T07:00:00Z", "id": "2pv" } ], "upcoming_verticals": [ { "effective_from": "2020-08-06T07:00:00Z", "free": 0, "timed": 0, "vertical_id": "2pv", "url": { "api": "https://api.viki.io/v4/verticals/2pv.json", "web": "https://www.viki.com/verticals/this_is_a_stub.json" }, "type": "vertical", "released_at": "2022-08-14T08:00:00Z", "id": "2pv" } ], "blocked": false, "blocking": { "geo": false, "paywall": false, "upcoming": false } } ```

This is different from, say, I am not a Robot, which the API reports to have 100% subtitle completion in Korean, but then several episodes come back with the subtitle_completions entry completely missing for Korean. In that case youtube-dl gently reports:

[download] Downloading video 23 of 33
[viki] 1123981v: Downloading video JSON
[viki] 1123981v: Downloading video streams JSON
WARNING: ko subtitles not available for 1123981v
firecrauter commented 3 years ago

Did you find a way to download the Viki videos at 720p ?, I would be very grateful if you told me

When I try to download with credentials/login; I get an error "HTTP Error 404: Not Found (caused by HTTPError 404: 'Not Found')