New Yankee Workshop later seasons fails with entitlement error even with cookie file

sjthespian commented 3 years ago

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.03.14
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

❯ youtube-dl -v --cookies cookies.txt -o 'The New Yankee Workshop S19E13 %(title)s.%(ext)s' https://www.thisoldhouse.com/season-19-new-yankee-workshop/21322864/s19-e13-window-bench
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--cookies', u'cookies.txt', u'-o', u'The New Yankee Workshop S19E13 %(title)s.%(ext)s', u'https://www.thisoldhouse.com/season-19-new-yankee-workshop/21322864/s19-e13-window-bench']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.03.14
[debug] Python version 2.7.16 (CPython) - Darwin-20.3.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[debug] Proxy map: {}
[ThisOldHouse] s19-e13-window-bench: Downloading webpage
[Zype] 5f973f23c08af40001e1bf05: Downloading JSON metadata
ERROR: This video requires a purchase entitlement in order to watch.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/zype.py", line 48, in _real_extract
    r'\.(?:js|html)\?', '.json?', url), video_id)['response']
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 899, in _download_json
    expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 877, in _download_json_handle
    expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 667, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 652, in _request_webpage
    raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 806, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 827, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/zype.py", line 52, in _real_extract
    e.cause.read().decode(), video_id)['message'], expected=True)
ExtractorError: This video requires a purchase entitlement in order to watch.

Description

Most of The New Yankee Workshop videos work fine once I am logged into their site, but the last three seasons are throwing the error ERROR: This video requires a purchase entitlement in order to watch. The videos play fine in the browser, and I am using the same cookie file that works with the first 16 seasons. I have also tried logging out and back in to regenerate the login tokens, that makes no difference.

This does require a login to access, I can share my cookie file directly with the devs and I am willing to work with the devs on debugging this in any way I can.

Below is an example of a working episode in verbose mode, up to the point where it starts the download.

❯ youtube-dl -v --cookies cookies.txt https://www.thisoldhouse.com/furniture/21053902/wall-mounted-tool-chest-new-yankee-workshop
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--cookies', u'cookies.txt', u'https://www.thisoldhouse.com/furniture/21053902/wall-mounted-tool-chest-new-yankee-workshop']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.03.14
[debug] Python version 2.7.16 (CPython) - Darwin-20.3.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[debug] Proxy map: {}
[ThisOldHouse] wall-mounted-tool-chest-new-yankee-workshop: Downloading webpage
[Zype] 5e0f6de0d7b3b70001f75c7a: Downloading JSON metadata
[Zype] 5e0f6de0d7b3b70001f75c7a: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://content-auso3.uplynk.com/4943d473bdd64af6ad3f50923ac42740/f.m3u8'
[hlsnative] Downloading m3u8 manifest
WARNING: hlsnative has detected features it does not support, extraction will be delegated to ffmpeg
[download] Destination: Wall-mounted Tool Chest _ New Yankee Workshop-5e0f6de0d7b3b70001f75c7a.mp4
[debug] ffmpeg command line: ffmpeg -y -loglevel verbose -headers 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.79 Safari/537.36
' -i 'https://content-auso3.uplynk.com/4943d473bdd64af6ad3f50923ac42740/f.m3u8' -c copy -f mp4 'file:Wall-mounted Tool Chest _ New Yankee Workshop-5e0f6de0d7b3b70001f75c7a.mp4.part'
ffmpeg version 4.3.2 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3.2_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[tcp @ 0x7f83d6806000] Starting connection attempt to 2600:1f16:e2:8101:39c2:45f1:cb6f:5ce port 443
[tcp @ 0x7f83d6806000] Successfully connected to 2600:1f16:e2:8101:39c2:45f1:cb6f:5ce port 443
[hls @ 0x7f83c4808200] Skip ('#EXT-X-VERSION:5')
[hls @ 0x7f83c4808200] Skip ('#UPLYNK-SEGMENT: 4943d473bdd64af6ad3f50923ac42740,00000000,segment')
[hls @ 0x7f83c4808200] Skip ('#UPLYNK-KEY:https://content-auso3.uplynk.com/check?b=4943d473bdd64af6ad3f50923ac42740&v=4943d473bdd64af6ad3f50923ac42740&pbs=')
[hls @ 0x7f83c4808200] Skip ('#EXT-X-PROGRAM-DATE-TIME:2021-03-21T02:08:21.801000+00:00')
[hls @ 0x7f83c4808200] HLS request for url 'https://x-default-stgec.uplynk.com/ausw/slices/494/7d44d8cebd184751b43edb3c8c8bdd44/4943d473bdd64af6ad3f50923ac42740/F00000000.ts?x=0&si=0', offset 0, playlist 0
[hls @ 0x7f83c4808200] Opening 'https://content-auso3.uplynk.com/check2?b=4943d473bdd64af6ad3f50923ac42740&v=4943d473bdd64af6ad3f50923ac42740&r=f&pbs=' for reading
[tcp @ 0x7f83b4625b80] Starting connection attempt to 2600:1f16:e2:8101:39c2:45f1:cb6f:5ce port 443
[tcp @ 0x7f83b4625b80] Successfully connected to 2600:1f16:e2:8101:39c2:45f1:cb6f:5ce port 443
[AVIOContext @ 0x7f83d44dde80] Statistics: 16 bytes read, 0 seeks
[hls @ 0x7f83c4808200] Opening 'crypto+https://x-default-stgec.uplynk.com/ausw/slices/494/7d44d8cebd184751b43edb3c8c8bdd44/4943d473bdd64af6ad3f50923ac42740/F00000000.ts?x=0&si=0' for reading

sjthespian commented 3 years ago

FWIW, I still see the same issue with 2021.03.25.

rfugina commented 3 years ago

I'm having the same issue with 2021.06.06. I've tried using a cookie file that includes cookies for thisoldhouse and zype. Older seasons work fine.

rfugina commented 3 years ago

youtube-dl -v --cookies cookies.txt https://www.thisoldhouse.com/season-17-new-yankee-workshop/21300923/s17-e11-kitchen-island [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'--cookies', u'cookies.txt', u'https://www.thisoldhouse.com/season-17-new-yankee-workshop/21300923/s17-e11-kitchen-island'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2021.06.06 [debug] Python version 2.7.17 (CPython) - Linux-4.15.0-143-generic-x86_64-with-Ubuntu-18.04-bionic [debug] exe versions: ffmpeg 4.2.1-static, ffprobe 3.4.8, phantomjs ., rtmpdump 2.4 [debug] Proxy map: {} [ThisOldHouse] s17-e11-kitchen-island: Downloading webpage [Zype] 5f8f4e7ff5dcd800017df89c: Downloading JSON metadata ERROR: This video requires a purchase entitlement in order to watch. Traceback (most recent call last): File "/usr/local/bin/youtube-dl/youtube_dl/extractor/zype.py", line 48, in _real_extract r'.(?:js|html)\?', '.json?', url), video_id)['response'] File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 899, in _download_json expected_status=expected_status) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 877, in _download_json_handle expected_status=expected_status) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 667, in _download_webpage_handle urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 652, in _request_webpage raise ExtractorError(errmsg, sys.exc_info()[2], cause=err) ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last): File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper return func(self, *args, **kwargs) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info ie_result = ie.extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract ie_result = self._real_extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/zype.py", line 52, in _real_extract e.cause.read().decode(), video_id)['message'], expected=True) ExtractorError: This video requires a purchase entitlement in order to watch.

ytdl-org / youtube-dl