10play.com.au not downloading

aschaft commented 2 years ago

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-u', 'PRIVATE', '-p', 'PRIVATE', '-i', '-f', 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best', 'https://10play.com.au/survivor-south-africa/episodes/season-9-return-of-the-outcasts/episode-1/tpv220717zymua']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.10.4 (CPython) - Linux-5.15.0-47-generic-x86_64-with-glibc2.35
[debug] exe versions: ffmpeg 4.4.2, ffprobe 4.4.2
[debug] Proxy map: {}
[TenPlay] tpv220717zymua: Downloading JSON metadata
ERROR: tpv220717zymua: Failed to parse JSON  (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 906, in _parse_json
    return json.loads(json_string)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 906, in _parse_json
    return json.loads(json_string)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/tenplay.py", line 41, in _real_extract
    data = self._download_json(
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 895, in _download_json
    res = self._download_json_handle(
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 881, in _download_json_handle
    return self._parse_json(
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 910, in _parse_json
    raise ExtractorError(errmsg, cause=ve)
youtube_dl.utils.ExtractorError: tpv220717zymua: Failed to parse JSON  (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

Trying on a aus based linode to download from 10play.com.au

cliveontoast commented 2 years ago

If you want to get into the weeds of pressing F12 on your browser, head over to the 'Network' Tab and when you hit the 'play' button you'll see an .m3u8

right click on that, and copy the URL

paste that into youtube-dl works well for me. Not ideal I know, but gets over the hump a bit.

youtube-dl.exe --verbose -u xxx -p xxx https://10play-vod.global.ssl.fastly.net/lots_of_letters_and_numbers/OTFP/more_letters_and_numbers/survsa-S9Ep1-1500000.m3u8

ends up with a file ~526mb.

I'm hiding the numbers and letters, cause i'm not sure if they're specifically for me, or a public URL.. sorry.

Vangelis66 commented 2 years ago

Several 10play.com.au-related open issues are to be found in the tracker currently:

https://github.com/ytdl-org/youtube-dl/issues?q=10play+is%3Aopen

most dating from 2021...

Troubleshooting this issue presents the following difficulties:

All 10play.com.au URLs are hard-blocked for those not residing Down Under 😞 (the sole current maintainer of youtube-dl is UK-based 😉 )
Some time ago, they made it mandatory to create an account with them in order to access (even free) episodes 👎
Several of their offerings are now behind full-blown DRM 😠

Trying on a aus based linode

Is that a VPS with an Australian IP? Please be aware that most media providers, especially in the recent years, tend to blacklist whole IP pools belonging to known datacenters, even when they're located in the coutnry/region where streaming is allowed... Make sure your VPS is indeed behind a whitelisted IP address as far as TenPlay are concerned (e.g. verify you can stream a show from a browser installed there...).

I believe the most relevant piece of info regarding this issue is to be found in #29638:

You need to provide credentials – but, YTDL also needs to be updated to take credentials for 10Play

Instead of passing -u+-p to youtube-dl, try to feed it cookies from a logged-in browser session instead (read the documentation on how to accomplish that...).

If you're willing to share account credentials (e.g. by creating a second "temp" account), it would greatly help the dev towards the goal of updating the app into accepting account credentials for TenPlay... 😜

Later addition: Relevant un-merged PR: #28728

If you're in a hurry, following advice from #29638 also applies:

You could try this fork, which works for me (...) : https://github.com/yt-dlp/yt-dlp

I live half a planet away from Australia, but I wanted to try 10play.com.au current support in yt-dlp; I had to

find a free+working+whitelisted AU HTTPS proxy to circumvent both "webpage" and "stream" hard geo-blocks
find leaked but still working 10play.com.au account credentials...

Trying latest master yt-dlp build, I think its tenplayIE works (at least for the show referenced in OP; i.e., it's not DRM'ed, as probably already made clear by @cliveontoast's contribution 😄 ) :

yt-dlp -v --proxy "https://proxyhost:proxyport" --ie tenplay -u <redacted> -p <redacted> -F "https://10play.com.au/survivor-south-africa/episodes/season-9-return-of-the-outcasts/episode-1/tpv220717zymua" => 

[debug] Command-line config: ['--ffmpeg-location', '..', '--downloader-args', 'ffmpeg:-v 8 -stats', '-v', '--proxy', 'https://proxyhost:proxyport', '--ie', 'tenplay', '-u', 'PRIVATE', '-p', 'PRIVATE', '-F', 'https://10play.com.au/survivor-south-africa/episodes/season-9-return-of-the-outcasts/episode-1/tpv220717zymua']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2022.09.14.041000 [1060f82] (win32_exe)
[debug] Python 3.7.9 (CPython 32bit) - Windows-Vista-6.0.6003-SP2
[debug] Checking exe version: "..\ffmpeg" -bsfs
[debug] Checking exe version: "..\ffprobe" -bsfs
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15.2, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {'http': 'https://proxyhost:proxyport', 'https': 'https://proxyhost:proxyport'}
[debug] Loaded 1 extractors
[debug] [TenPlay] Extracting URL: https://10play.com.au/survivor-south-africa/episodes/season-9-return-of-the-outcasts/episode-1/tpv220717zymua
[TenPlay] tpv220717zymua: Downloading JSON metadata
[TenPlay] tpv220717zymua: Getting bearer token
[TenPlay] tpv220717zymua: Downloading video JSON
[TenPlay] tpv220717zymua: Downloading webpage
[TenPlay] tpv220717zymua: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7),vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for 6309654142112:
ID   EXT RESOLUTION FPS |   FILESIZE   TBR PROTO | VCODEC        VBR ACODEC    ABR
----------------------------------------------------------------------------------
722  mp4 480x270     25 | ~232.84MiB  722k m3u8  | avc1.4d4015  722k mp4a.40.20k
931  mp4 640x360     25 | ~300.22MiB  931k m3u8  | avc1.4d401e  931k mp4a.40.20k
1695 mp4 960x540     25 | ~546.60MiB 1695k m3u8  | avc1.4d401f 1695k mp4a.40.20k

dirkf commented 2 years ago

Thanks for that very thorough response.

The version of the tenplay.py extractor in this zip archive includes the yt-dlp changes plus some of mine.

As I can only get this

This video is not available from your location due to geo restriction
This video is available in Australia.
You might want to use a VPN or a proxy server (with --proxy) to workaround.

without going to the lengths (1 & 2, as listed) above, I encourage any savvy 10Play subscriber with the sort of installation that allows individual files to be replaced to try the zipped module instead of the shipped extractor/tenplay.py.

Although this might be a duplicate of the closed #28641, any discussion can continue here.

Vangelis66 commented 2 years ago

... dirkf's patch appears to work as expected (tested on the show in the OP):

youtube-dl -v --proxy "https://proxyhost:proxyport" -u <redacted> -p <redacted> -F "https://10play.com.au/survivor-south-africa/episodes/season-9-return-of-the-outcasts/episode-1/tpv220717zymua" => 

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--proxy', 'https://proxyhost:proxyport', '-u', 'PRIVATE', '-p', 'PRIVATE', '-F', 'https://10play.com.au/survivor-south-africa/episodes/season-9-return-of-the-outcasts/episode-1/tpv220717zymua']
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2022.09.16.1919
[debug] Python version 3.4.4 (CPython) - Windows-Vista-6.0.6003-SP2
[debug] exe versions: ffmpeg 4.4.1, ffprobe 4.4.1, phantomjs 2.1.1, rtmpdump 2.4

[debug] Proxy map: {'https': 'https://proxyhost:proxyport', 'http': 'https://proxyhost:proxyport'}
[TenPlay] tpv220717zymua: Downloading JSON metadata
[TenPlay] tpv220717zymua: Getting bearer token
[TenPlay] tpv220717zymua: Downloading video JSON
[TenPlay] tpv220717zymua: Downloading webpage
[TenPlay] tpv220717zymua: Downloading m3u8 information
[info] Available formats for 6309654142112:
format code  extension  resolution note
722          mp4        480x270     722k , avc1.4d4015, 25.0fps, mp4a.40.2
931          mp4        640x360     931k , avc1.4d401e, 25.0fps, mp4a.40.2
1695         mp4        960x540    1695k , avc1.4d401f, 25.0fps, mp4a.40.2 (best)

and

youtube-dl --proxy "https://proxyhost:proxyport" -u <redacted> -p <redacted> "https://10play.com.au/survivor-south-africa/episodes/season-9-return-of-the-outcasts/episode-1/tpv220717zymua" --hls-prefer-native => 

[TenPlay] tpv220717zymua: Downloading JSON metadata
[TenPlay] tpv220717zymua: Getting bearer token
[TenPlay] tpv220717zymua: Downloading video JSON
[TenPlay] tpv220717zymua: Downloading webpage
[TenPlay] tpv220717zymua: Downloading m3u8 information
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 441
[download] Destination: Survivor South Africa - S9 Ep. 1-6309654142112.mp4
[download]   0.2% of ~542.88MiB at 160.38KiB/s ETA 01:51:13

(my free AU proxy has, expectedly, slow speeds 😜 ... )

ytdl-org / youtube-dl