Open CollinChaffin opened 8 years ago
Perhaps a title of "Site Request: cwseed.com" might speed adoption?
Sorry and thanks!
Notes:
Click on an episode, such as OC Pilot, rendered the following link:
http://www.cwseed.com/shows/the-oc/pilot/?play=00f4dacd-d0b4-4fc5-99ac-0a59142dea42
Appears video metadata is coming from this location:
http://metaframe.digitalsmiths.tv/v2/CWtv/assets/00f4dacd-d0b4-4fc5-99ac-0a59142dea42/partner/53?format=json
Example JSON output:
{
"videos":{
"ds900":{
"length":2717,
"bitrate":"900",
"uri":"rtmpe://wbworldtv.fcod.llnwd.net/a2246/o23/mp4:cwtv/videos/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_900kbps.mp4"
},
"ds300":{
"length":2717,
"bitrate":"300",
"uri":"rtmpe://wbworldtv.fcod.llnwd.net/a2246/o23/mp4:cwtv/videos/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_300kbps.mp4"
},
"ds500":{
"length":2717,
"bitrate":"500",
"uri":"rtmpe://wbworldtv.fcod.llnwd.net/a2246/o23/mp4:cwtv/videos/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_500kbps.mp4"
},
"ds700":{
"length":2717,
"bitrate":"700",
"uri":"rtmpe://wbworldtv.fcod.llnwd.net/a2246/o23/mp4:cwtv/videos/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_700kbps.mp4"
}
},
"fullPlayback":"true",
"videoBreaks":[
],
"expireTime":"2016-11-02T06:59:00+00:00",
"adInfo":{
"seriesSymbol":"TOC",
"genre_cgid":"030205"
},
"assetFields":{
"rating":"0.0000",
"smoothStreamingUrl":"http://cwsmooth-vod.hss.adaptive.level3.net/b08b1901-2a8d-4208-ad20-a8124e1d35c9/TheOC-101-Premiere-475197.ism/Manifest",
"localPodEnabled":"false",
"showSlug":"the-oc",
"duration":"2717",
"isEmbeddable":"false",
"episodeNumber":"1",
"title":"Pilot",
"assetId":"105794",
"seriesName":"CW Seed - The O.C.",
"dartZone":"cwseedtheoc",
"dartSite":"cwtv.fullstream",
"videoCDN":"LIMELIGHT",
"seriesVideoSlate":"false",
"xboxSmartglassEnabled":"false",
"contentType":"Web Originals",
"description":"Welcome to the O.C., bitch! A troubled teen moves in with his lawyer's family. Newport Beach will never be the same!",
"seoShowTitle":"the-oc-pilot",
"domainWhitelist":"digitalsmiths.com,cwtv.com,cwseed.com,fuiszmedia.com",
"UnicornCcUrl":"http://api.digitalsmiths.tv/cc/x4abd9107/2015/08/13/TheOC-101-Premiere-475197_a59cdfc3b.dfxp",
"seasonNumber":"1",
"popularity":"0",
"seasonName":"1",
"housePromoId":"CSD"
},
"adCues":[
{
"duration":"120.00",
"endTime":"731.03",
"mobileDuration":"120.00",
"startTime":"731.00"
},
{
"duration":"120.00",
"endTime":"1581.01",
"mobileDuration":"120.00",
"startTime":"1581.01"
},
{
"duration":"120.00",
"endTime":"1918.02",
"mobileDuration":"120.00",
"startTime":"1918.02"
},
{
"duration":"120.00",
"endTime":"2238.00",
"mobileDuration":"120.00",
"startTime":"2238.00"
},
{
"duration":"120.00",
"endTime":"2676.01",
"mobileDuration":"120.00",
"startTime":"2676.01"
}
],
"startTime":"2015-10-31T08:01:00+00:00",
"images":{
"cwtv720x400":{
"width":720,
"uri":"http://media.cwtv.com/cwtv/digital-smiths/thumbs/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_CWtv_720x400.jpg",
"height":400
},
"cwtv210x115":{
"width":210,
"uri":"http://media.cwtv.com/cwtv/digital-smiths/thumbs/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_CWtv_210x115.jpg",
"height":115
},
"cwtv1920x1080":{
"width":1920,
"uri":"http://media.cwtv.com/cwtv/digital-smiths/thumbs/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_CWtv_1920x1080.jpg",
"height":1080
},
"cwtv640x360":{
"width":640,
"uri":"http://media.cwtv.com/cwtv/digital-smiths/thumbs/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_CWtv_640x360.jpg",
"height":360
},
"cwtv640x480":{
"width":640,
"uri":"http://media.cwtv.com/cwtv/digital-smiths/thumbs/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_CWtv_640x480.jpg",
"height":480
},
"cwtv141x106":{
"width":141,
"uri":"http://media.cwtv.com/cwtv/digital-smiths/thumbs/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_CWtv_141x106.jpg",
"height":106
},
"cwtv141x79":{
"width":141,
"uri":"http://media.cwtv.com/cwtv/digital-smiths/thumbs/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_CWtv_141x79.jpg",
"height":79
},
"cwtv609x335":{
"width":609,
"uri":"http://media.cwtv.com/cwtv/digital-smiths/thumbs/2015/07/30/TheOC-101-Premiere-475197_a64e25b16_CWtv_609x335.jpg",
"height":335
}
},
"assetRights":{
"blockAds":"false"
},
"guid":"00f4dacd-d0b4-4fc5-99ac-0a59142dea42"
}
Interesting! So it redirects to wbworldtv.fcod.llnwd.net.
It uses Limelight through RTMP streaming (hence the RTMPE for RTMP encryption). There's extractors for RTMP and I've verified through rtmpdump that it can download them. Just need to clobb something together.
Wow awesome David!
Hey David, were you ever able to figure this one out? I just tried the latest version and it still doesn't recognize it. Thanks!
i created the extractor for cwtv and cwseed. i find a way to extract m3u8 formats but the rtmpe urls didin't work for me(even with flash player argument).
I've verified through rtmpdump that it can download them.
@davidjameshowell if you can share the rtmpdump command you use to get cwtv or cwseed rtmpe streams so i will fix rtmpe urls and push the extractor.
episode downloading will be supported in the next version. i will keep this issue open as the url you've request is not supported yet(show pages).
Wow excellent! So for the next version does that mean you already got the code done for the show pages in addition to what you already did for the overall extractor? What is the build schedule before you push it and it's recompiled into the binary update release?
So for the next version does that mean you already got the code done for the show pages
no i'm not working on show pages. it should be simple to add support for them but i'm working on other sites.
What is the build schedule before you push it and it's recompiled into the binary update release?
i don't know when the next version will be released(most of the time there is at least one release per weak).
Version 2016.01.15 includes support for single episodes. See our FAQ if you need help updating.
Thanks I know how to update I was asking how often these build changes roll into the public "updates". You guys did awesome job can't wait to see the code for the show pages as a good example!
% ./youtube-dl --verbose 'http://www.cwseed.com/shows/mortal-kombat-x/round-4/?play=0470e385-4df8-4f16-be88-4bc5220dbd80'
[debug] System config: []
[debug] User config: ['--netrc', '-t', '--restrict-filenames', '--no-part', '--prefer-free-formats', '--add-metadata']
[debug] Command-line args: ['--verbose', 'http://www.cwseed.com/shows/mortal-kombat-x/round-4/?play=0470e385-4df8-4f16-be88-4bc5220dbd80']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.02.01
[debug] Python version 3.5.1 - Linux-4.3.3-3-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 2.8.5, ffprobe 2.8.5, rtmpdump 2.4
[debug] Proxy map: {}
[CWTV] 0470e385-4df8-4f16-be88-4bc5220dbd80: Downloading JSON metadata
[CWTV] 0470e385-4df8-4f16-be88-4bc5220dbd80: Downloading m3u8 information
[debug] Invoking downloader on 'http://hlsioscwtv.warnerbros.com/hls/2016/01/29/MortalKombat-ChasingTheCup-104-CW-Stereo-NoWM-Compressor_a6285a1e2_2100kbps/MortalKombat-ChasingTheCup-104-CW-Stereo-NoWM-Compressor_a6285a1e2_2100kbps.m3u8'
[download] Destination: Round_4-0470e385-4df8-4f16-be88-4bc5220dbd80.mp4
[debug] ffmpeg command line: ffmpeg -y -headers 'Accept-Language: en-us,en;q=0.5
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
' -i http://hlsioscwtv.warnerbros.com/hls/2016/01/29/MortalKombat-ChasingTheCup-104-CW-Stereo-NoWM-Compressor_a6285a1e2_2100kbps/MortalKombat-ChasingTheCup-104-CW-Stereo-NoWM-Compressor_a6285a1e2_2100kbps.m3u8 -c copy -f mp4 -bsf:a aac_adtstoasc file:Round_4-0470e385-4df8-4f16-be88-4bc5220dbd80.mp4
ffmpeg version 2.8.5 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 5.3.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdcadec --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100
http://hlsioscwtv.warnerbros.com/hls/2016/01/29/MortalKombat-ChasingTheCup-104-CW-Stereo-NoWM-Compressor_a6285a1e2_2100kbps/MortalKombat-ChasingTheCup-104-CW-Stereo-NoWM-Compressor_a6285a1e2_2100kbps.m3u8: Server returned 401 Unauthorized (authorization failed)
ERROR: ffmpeg exited with code 1
File "/usr/lib/python3.5/runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "./youtube-dl/__main__.py", line 19, in <module>
youtube_dl.main()
File "./youtube-dl/youtube_dl/__init__.py", line 411, in main
_real_main(argv)
File "./youtube-dl/youtube_dl/__init__.py", line 401, in _real_main
retcode = ydl.download(all_urls)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 1690, in download
url, force_generic_extractor=self.params.get('force_generic_extractor', False))
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 677, in extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 722, in process_ie_result
return self.process_video_result(ie_result, download=download)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 1360, in process_video_result
self.process_info(new_info)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 1622, in process_info
success = dl(filename, info_dict)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 1564, in dl
return fd.download(name, info)
File "./youtube-dl/youtube_dl/downloader/common.py", line 343, in download
return self.real_download(filename, info_dict)
File "./youtube-dl/youtube_dl/downloader/hls.py", line 76, in real_download
self.report_error('%s exited with code %d' % (ffpp.basename, retval))
File "./youtube-dl/youtube_dl/downloader/common.py", line 156, in report_error
self.ydl.report_error(*args, **kargs)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 541, in report_error
self.trouble(error_message, tb)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 503, in trouble
tb_data = traceback.format_list(traceback.extract_stack())
@shizeeg edit your comment with the output of the youtube-dl --verbose 'http://www.cwseed.com/shows/mortal-kombat-x/round-4/?play=0470e385-4df8-4f16-be88-4bc5220dbd80'
Hi looks like this was closed but does not still appear to work to download all episodes from a show's main page listing the multiple episode files. How would one be able to use youtube-dl to cache like 4 seasons without having to open and paste 50+ individual links to youtube-dl? Thanks!
@CollinChaffin please open a new issue for show pages.
I see this was re-opened, do I still need to open a new issue rather than allow this issue to encompass both the completed individual episodes as well as the not yet completed main show index page?
Sergey is there a similar working extractor for parent "show" pages that then recurses the RTMP episodes that you could point me to as an example? I would like to see if I might be able to contribute and clob something together based on something similar. Thanks!
>youtube-dl -v https://cwseed.com/shows/vixen/s2-episode-one/?play=74315115-2f11-4448-abc9-5a1a5324e506
[debug] System config: []
[debug] User config: ['--sub-lang', 'ru,en']
[debug] Command-line args: ['-v', 'https://cwseed.com/shows/vixen/s2-episode-one/?play=74315115-2f11-4448-abc9-5a1a5324e506']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.10.16
[debug] Python version 3.4.3 - Linux-4.1.34-gentoo-nvidia-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E8400_@_3.00GHz-with-gentoo-2.2
[debug] exe versions: ffmpeg 3.1.4, ffprobe 3.1.4, rtmpdump 2.4
[debug] Proxy map: {}
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Downloading JSON metadata
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Downloading m3u8 information
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Downloading JSON metadata
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Checking http-5128 URL
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: http-5128 URL is invalid, skipping
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Checking http-264 URL
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: http-264 URL is invalid, skipping
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Checking http-664 URL
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: http-664 URL is invalid, skipping
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Checking http-8628 URL
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: http-8628 URL is invalid, skipping
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Checking http-464 URL
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: http-464 URL is invalid, skipping
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Checking http-6628 URL
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: http-6628 URL is invalid, skipping
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Checking http-1296 URL
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: http-1296 URL is invalid, skipping
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: Checking http-3596 URL
[CWTV] 74315115-2f11-4448-abc9-5a1a5324e506: http-3596 URL is invalid, skipping
[debug] Invoking downloader on 'http://hlsioscwtv.warnerbros.com/hls/2016/10/05/VIX-Episode201-CWSeed_b1d760ae2_2100kbps/VIX-Episode201-CWSeed_b1d760ae2_2100kbps.m3u8'
[download] Destination: S2 - Episode One-74315115-2f11-4448-abc9-5a1a5324e506.mp4
[debug] ffmpeg command line: ffmpeg -y -headers 'Accept-Language: en-us,en;q=0.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
' -i http://hlsioscwtv.warnerbros.com/hls/2016/10/05/VIX-Episode201-CWSeed_b1d760ae2_2100kbps/VIX-Episode201-CWSeed_b1d760ae2_2100kbps.m3u8 -c copy -f mp4 -bsf:a aac_adtstoasc 'file:S2 - Episode One-74315115-2f11-4448-abc9-5a1a5324e506.mp4.part'
ffmpeg version 3.1.4 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.9.3 (Gentoo 4.9.3 p1.5, pie-0.6.4)
configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.1.4/html --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-march=core2 -O2 -pipe' --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=oss --disable-indev=jack --disable-outdev=oss --enable-version3 --enable-bzlib --disable-runtime-cpudetect --disable-debug --enable-gcrypt --disable-gnutls --enable-gmp --enable-gpl --enable-hardcoded-tables --enable-iconv --disable-lzma --enable-network --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl --disable-vaapi --enable-vdpau --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --disable-libpulse --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libcelt --disable-libgme --disable-libgsm --disable-mmal --disable-libmodplug --disable-libopus --disable-libilbc --enable-librtmp --disable-libssh --disable-libschroedinger --disable-libspeex --enable-libvorbis --disable-libvpx --enable-libzvbi --disable-libbs2b --disable-chromaprint --disable-libebur128 --disable-libflite --disable-frei0r --disable-libfribidi --enable-fontconfig --disable-ladspa --enable-libass --enable-libfreetype --disable-librubberband --disable-libzimg --disable-libsoxr --enable-pthreads --disable-libvo-amrwbenc --enable-libmp3lame --disable-libfaac --disable-libkvazaar --disable-nvenc --disable-libopenh264 --disable-libsnappy --disable-libtheora --disable-libtwolame --disable-libwavpack --disable-libwebp --disable-libx264 --disable-libx265 --enable-libxvid --disable-amd3dnow --disable-amd3dnowext --disable-aesni --disable-avx --disable-avx2 --disable-fma3 --disable-fma4 --disable-sse42 --disable-xop --cpu=core2 --disable-doc --disable-htmlpages --enable-manpages
libavutil 55. 28.100 / 55. 28.100
libavcodec 57. 48.101 / 57. 48.101
libavformat 57. 41.100 / 57. 41.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 47.100 / 6. 47.100
libavresample 3. 0. 0 / 3. 0. 0
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100
[http @ 0x1abde70] HTTP error 403 Forbidden
http://hlsioscwtv.warnerbros.com/hls/2016/10/05/VIX-Episode201-CWSeed_b1d760ae2_2100kbps/VIX-Episode201-CWSeed_b1d760ae2_2100kbps.m3u8: Server returned 403 Forbidden (access denied)
ERROR: ffmpeg exited with code 1
File "/usr/bin/youtube-dl", line 11, in <module>
sys.exit(main())
File "/usr/lib64/python3.4/site-packages/youtube_dl/__init__.py", line 445, in main
_real_main(argv)
File "/usr/lib64/python3.4/site-packages/youtube_dl/__init__.py", line 435, in _real_main
retcode = ydl.download(all_urls)
File "/usr/lib64/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 1791, in download
url, force_generic_extractor=self.params.get('force_generic_extractor', False))
File "/usr/lib64/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 705, in extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/usr/lib64/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 751, in process_ie_result
return self.process_video_result(ie_result, download=download)
File "/usr/lib64/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 1435, in process_video_result
self.process_info(new_info)
File "/usr/lib64/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 1699, in process_info
success = dl(filename, info_dict)
File "/usr/lib64/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 1641, in dl
return fd.download(name, info)
File "/usr/lib64/python3.4/site-packages/youtube_dl/downloader/common.py", line 353, in download
return self.real_download(filename, info_dict)
File "/usr/lib64/python3.4/site-packages/youtube_dl/downloader/external.py", line 43, in real_download
self.get_basename(), retval))
File "/usr/lib64/python3.4/site-packages/youtube_dl/downloader/common.py", line 162, in report_error
self.ydl.report_error(*args, **kargs)
File "/usr/lib64/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 568, in report_error
self.trouble(error_message, tb)
File "/usr/lib64/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 530, in trouble
tb_data = traceback.format_list(traceback.extract_stack())
Tried several urls, all the same: HTTP error 403 Forbidden.
cwseed episode links work for me. When I get 403s they are e. g. due to country restrictions.
CW TV show pages still doesn't work for me. But single episode pages work fine. How is it going with support for Show pages on CW TV/Seed?
Hi, I'd like to vote to add the following site which does not appear to currently work but does list all videos on one page, etc. so given other supported sites doesn't look like it would be very different:
Site: http://www.cwseed.com
Example of series page: http://www.cwseed.com/shows/the-oc