search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.3k stars 9.95k forks source link

Support request for ert.gr #15960

Open Hackerpcs opened 6 years ago

Hackerpcs commented 6 years ago

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.03.20.

Before submitting an issue make sure you have:

What is the purpose of your issue?

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output:

youtube-dl.exe -v http://webtv.ert.gr/promo-ert1/22mar2018-the-mitsi-show/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'http://webtv.ert.gr/promo-ert1/22mar2018-the-mitsi-show/']
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2018.03.20
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-90061-gfb580731c1, ffprobe N-90061-gfb580731c1, phantomjs 2.1.1
[debug] Proxy map: {'https': 'https://127.0.0.1:3128', 'http': 'http://127.0.0.1:3128', 'ftp': 'ftp://127.0.0.1:3128'}
[generic] 22mar2018-the-mitsi-show: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 22mar2018-the-mitsi-show: Downloading webpage
[generic] 22mar2018-the-mitsi-show: Extracting information
ERROR: Unsupported URL: http://webtv.ert.gr/promo-ert1/22mar2018-the-mitsi-show/
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\YoutubeDL.py", line 785, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\common.py", line 440, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\generic.py", line 3161, in _real_extract
youtube_dl.utils.UnsupportedError: Unsupported URL: http://webtv.ert.gr/promo-ert1/22mar2018-the-mitsi-show/

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Web TV site of state Greek TV

Vangelis66 commented 6 years ago

Almost all of ERT VOD uses AppleHLS (geo-fencing possible for some content)...

22mar2018-the-mitsi-show => http://ep.ert.gr:1935/vodedge/_definst_/mp4:dvrorigin/2018/tv/the-mitsi-show/20180322-the-mitsi-show.mp4/playlist.m3u8

Hackerpcs commented 6 years ago

I know, I can grab the HLS with VideoDownload Helper and then feed it, I'm asking for support to mass-download links or play directly with a simple "mpv url"

clavisound commented 2 years ago

They changed to dash.

example: URL: https://www.ertflix.gr/series/ser.3448-monogramma

MPD: https://mediaserve.ert.gr/bpk-vod/vodext/default/monogramma-giorgos-makropoulos/monogramma-giorgos-makropoulos/index.mpd

Vangelis66 commented 2 years ago

@clavisound: You resurrected an issue from 3 1/2 years ago, so yeah, a lot has changed since then... 😜 ERT is the Greek National TV Broadcaster; what used to be webtv.ert.gr, where LiveTV and VOD was served from, has become ERTFLIX more than a year ago, now hosted on www.ertflix.gr, with an overhauled site design/embedded webplayer/underlying infrastructure/etc. ...

The change of service hostname probably makes this old issue a deprecated one, that should be closed... Given that youtube-dl is currently in an uncertain, frozen, state (let's call it "abandonware"), I would advise you to direct www.ertflix.gr support requests to one of the currently maintained forks...

Some additional info:

  1. Expect some content to be reserved for Greek IPs, only.
  2. At the time of this writing, no mandatory sign-in is enforced to watch VODs; subject to change.
  3. The new web player infrastructure supports EME/DRM, but DRM is not currently enforced; subject to change, too...

They changed to dash.

AppleHLS streams are also available, but not the default for desktop browsers (perhaps applicable on desktop Safari?); the playlist API URI for the programme you referenced is: 

https://api.app.ertflix.gr/v1/Player/AcquireContent?platformCodename=www&codename=monogramma-giorgos-markopoylos

(JSON response), from which I can locate:

https://mediaserve.ert.gr/bpk-vod/vodext/default/monogramma-giorgos-makropoulos/monogramma-giorgos-makropoulos/index.m3u8

#EXTM3U
#EXT-X-VERSION:7
## Created with Broadpeak BkS350 Origin Packager  (version=1.11.1-24062)

# AUDIO groups
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-96",NAME="audio",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-128",NAME="audio",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-192",NAME="audio",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-256",NAME="audio",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aacl-320",NAME="audio",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"

# variants
#EXT-X-STREAM-INF:BANDWIDTH=633000,CODECS="mp4a.40.2,avc1.4D401E",RESOLUTION=640x360,FRAME-RATE=25,VIDEO-RANGE=SDR,AUDIO="audio-aacl-96",CLOSED-CAPTIONS=NONE
monogramma-giorgos-makropoulos-audio=96000-video=500233.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1197000,CODECS="mp4a.40.2,avc1.4D401E",RESOLUTION=768x432,FRAME-RATE=25,VIDEO-RANGE=SDR,AUDIO="audio-aacl-128",CLOSED-CAPTIONS=NONE
monogramma-giorgos-makropoulos-audio=128000-video=1000582.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2325000,CODECS="mp4a.40.2,avc1.64001F",RESOLUTION=1280x720,FRAME-RATE=25,VIDEO-RANGE=SDR,AUDIO="audio-aacl-192",CLOSED-CAPTIONS=NONE
monogramma-giorgos-makropoulos-audio=192000-video=2001003.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3453000,CODECS="mp4a.40.2,avc1.640028",RESOLUTION=1920x1080,FRAME-RATE=25,VIDEO-RANGE=SDR,AUDIO="audio-aacl-256",CLOSED-CAPTIONS=NONE
monogramma-giorgos-makropoulos-audio=256000-video=3001122.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=4582000,CODECS="mp4a.40.2,avc1.640028",RESOLUTION=1920x1080,FRAME-RATE=25,VIDEO-RANGE=SDR,AUDIO="audio-aacl-320",CLOSED-CAPTIONS=NONE
monogramma-giorgos-makropoulos-audio=320000-video=4002227.m3u8

# variants
#EXT-X-STREAM-INF:BANDWIDTH=102000,CODECS="mp4a.40.2",AUDIO="audio-aacl-96"
monogramma-giorgos-makropoulos-audio=96000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=136000,CODECS="mp4a.40.2",AUDIO="audio-aacl-128"
monogramma-giorgos-makropoulos-audio=128000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=204000,CODECS="mp4a.40.2",AUDIO="audio-aacl-192"
monogramma-giorgos-makropoulos-audio=192000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=272000,CODECS="mp4a.40.2",AUDIO="audio-aacl-256"
monogramma-giorgos-makropoulos-audio=256000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=340000,CODECS="mp4a.40.2",AUDIO="audio-aacl-320"
monogramma-giorgos-makropoulos-audio=320000.m3u8

# keyframes
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=67000,CODECS="avc1.4D401E",RESOLUTION=640x360,VIDEO-RANGE=SDR,URI="keyframes/monogramma-giorgos-makropoulos-video=500233.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=133000,CODECS="avc1.4D401E",RESOLUTION=768x432,VIDEO-RANGE=SDR,URI="keyframes/monogramma-giorgos-makropoulos-video=1000582.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=266000,CODECS="avc1.64001F",RESOLUTION=1280x720,VIDEO-RANGE=SDR,URI="keyframes/monogramma-giorgos-makropoulos-video=2001003.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=398000,CODECS="avc1.640028",RESOLUTION=1920x1080,VIDEO-RANGE=SDR,URI="keyframes/monogramma-giorgos-makropoulos-video=3001122.m3u8"
Hackerpcs commented 2 years ago

MPD can be downloaded with straight ffmpeg, get the mpd from the browser's console

ffmpeg -i <mpd url> -map 0:3 -map 0:9 -c copy download.mkv

map arguments are needed because I've seen that ffmpeg doesn't select the best options, check the available ones with ffprobe

ffprobe <mpd url>

and choose the last video and audio streams that are the best

clavisound commented 2 years ago

Thank you about the ffmpeg tip. To me the "best" is the lowest acceptable size to my eyes and ears :-)

Personally I don't have a problem, but it will be nice if youtube-dl URL automagically fetches the .mpd from the URL e.t.c.

This script is fine with me.

#!/bin/bash
YTBIN=${YTBIN:-"youtube-dl"}
LIMITRATE=${LIMITRATE:-"-r 150k"}
URL=$1

FILE=$(echo $URL | sed -e 's+.*default/++g' -e 's+/.*++')
wget $URL -O $FILE.mpd

VIDEOQ=$(grep width=.768 $FILE.mpd | gawk -F'"' '{ print $2 }')
AUDIOQ="audio=128000"

for STREAM in $VIDEOQ $AUDIOQ
do
    echo $YTBIN -f $STREAM $LIMITRATE $URL -o $FILE.$STREAM
    $YTBIN -f $STREAM $LIMITRATE $URL -o $FILE.$STREAM
done

ffmpeg -i $FILE.$VIDEOQ -i $FILE.$AUDIOQ -c copy -y $FILE.mp4

ffmpeg fails. Maybe I have disabled something. Will check.

ffprobe -i https://mediaserve.ert.gr/bpk-vod/vodext/default/monogramma-giorgos-makropoulos/monogramma-giorgos-makropoulos/index.mpd
ffprobe version 3.2.4 Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 5.5.0 (GCC)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/doc/ffmpeg-3.2.4/html --mandir=/usr/man --disable-debug --enable-shared --disable-static --enable-gpl --enable-version3 --enable-avresample --arch=x86_64 --enable-nonfree --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gnutls --enable-libass --enable-libcaca --enable-libcdio --enable-libcelt --enable-librtmp --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-opengl --enable-libopenjpeg --enable-libpulse --enable-libsmbclient --enable-libwavpack --enable-x11grab
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
https://mediaserve.ert.gr/bpk-vod/vodext/default/monogramma-giorgos-makropoulos/monogramma-giorgos-makropoulos/index.mpd: Input/output error
Hackerpcs commented 2 years ago

That's a very old ffmpeg from 2017 probably without mpd support, update it to the latest version

Vangelis66 commented 2 years ago

get the mpd from the browser's console

For Chromium-based browsers, below extension is also useful:

https://chrome.google.com/webstore/detail/mpd-detector/lpoohbdbmggiknlpcmhhdkpaclfcdapk

map arguments are needed, because I've seen that ffmpeg doesn't select the best options

This has been a long-standing issue 😠 ever since the ffmpeg devs implemented the dashdemuxer some years ago, and it's infuriating 💢 they haven't acknowledged it yet, hence nothing is in the works to address it... Third-party apps (yt-dl, streamlink) with their own MPD parser are needed to mitigate this ffmpeg bug...

and choose the last video and audio streams that are the best

I usually watch ERT VODs on my 15.6in laptop, anything above 720p (video) and 128k (audio) is a waste of bandwidth (to fetch) and disk space (to store); if you like to watch on a wide-screen SmartTV, bigger is better, of course 😉 ...

ffmpeg fails. Maybe I have disabled something. 

ffprobe version 3.2.4 Copyright (c) 2007-2017 the FFmpeg developers

You're on a Linux distro, sadly I know little about Linux... Try to upgrade your FFmpeg build to the 4.4 release branch (or higher), make sure the native DASH demuxer has been enabled during compilation of that build (a third-party XML parser lib like --enable-libxml2 might be needed for that...).

Hackerpcs commented 2 years ago

Personally I download to archive so I select the best quality anyway. Also I download when I can't record the program directly from digital terrestrial TV which I usually do and the resulting 1080i file is more than twice the size from the best ERT mpd quality

Hackerpcs commented 2 years ago

Issue should be closed, URL is dead

As you can see from the 2021 replies above the site just changed to ertflix.gr, e.g.

https://www.ertflix.gr/vod/vod.160900-e-tourta-tes-mamas-17