Closed cannamaier closed 5 years ago
DerBunteBall
commented
5 years ago Can agree. I accidentally opened also an issue in the same time window. See #18500.
The portal cahnged totally today or yesterday. The extractor seems to need rework again. As we know TVNow often changes.
Best Regards
cannamaier
commented
5 years ago You don't need a account for download, this was in the version before also.
fabki
commented
5 years ago Should not be a major re-programming part.
API changed from https://api.tvnow.de/v3/ to https://apigw.tvnow.de/
revunix
commented
5 years ago @fabki good to know, have you already tested it?
fabki
commented
5 years ago not yet, maybe after work on the late hours today.
revunix
commented
5 years ago That would be nice. 👊🏻
fabki
commented
5 years ago Hmm seems not to be so easy as i thought.
Only free content can be grabed with new api, but script also needs modification (i'm not good with python :( ). The DRM-free "Premium-Content" can not be accessed without credentials. (i.e. older episodes of free content)
The "good" one: Simple need to grab the ID - i.e. 1333774 -> https://apigw.tvnow.de/module/player/1333774 - and getting all informations that we need to format and download.
revunix
commented
5 years ago I tried it out, the dashhd works ... and i can it download with the old version of youtube-dl. Thats good so far. Its more work but it works for now.
EDIT:
The funny part is... jDownload can download the Premium-Content w/o a Account. wtf? how...
i decoded the jdownloader .class and found a new API: https://api.tvnow.de/v3/movies/ID (DRM Content inc.)
package jd.plugins.hoster;
import java.util.LinkedHashMap;
import java.util.Locale;
import jd.PluginWrapper;
import jd.http.Browser;
import jd.http.RequestHeader;
import jd.http.URLConnectionAdapter;
import jd.parser.Regex;
import jd.plugins.DownloadLink;
import jd.plugins.DownloadLink.AvailableStatus;
import jd.plugins.FilePackage;
import jd.plugins.HostPlugin;
import jd.plugins.PluginException;
import jd.plugins.PluginForHost;
import jd.plugins.components.MediathekHelper;
import jd.plugins.download.DownloadInterface;
import org.appwork.utils.StringUtils;
import org.appwork.utils.formatter.TimeFormatter;
import org.jdownloader.downloader.hls.HLSDownloader;
import org.jdownloader.plugins.components.config.MediathekProperties;
import org.jdownloader.plugins.components.hls.HlsContainer;
import org.jdownloader.plugins.config.PluginConfigInterface;
import org.jdownloader.plugins.config.PluginJsonConfig;
import org.jdownloader.scripting.JavaScriptEngineFactory;
@HostPlugin(revision="$Revision: 40168 $", interfaceVersion=3, names={"tvnow.de"}, urls={"https?://(?:www\\.)?(?:nowtv|tvnow)\\.(?:de|ch)/[a-z0-9\\-]+/[a-z0-9\\-]+/.+"})
public class TvnowDe
extends PluginForHost
{
private static final String TYPE_GENERAL_ALRIGHT = "https?://(?:www\\.)?(?:nowtv|tvnow)\\.(?:de|ch)/[^/]+/[a-z0-9\\-]+/[^/\\?]+";
public static final String API_BASE = "https://api.tvnow.de/v3";
public static final String CURRENT_DOMAIN = "tvnow.de";
public TvnowDe(PluginWrapper wrapper)
{
super(wrapper);
}
private LinkedHashMap<String, Object> entries = null;
public static Browser prepBR(Browser br) {
br.getHeaders().put("Accept", "application/json, text/plain, */*");
br.setAllowedResponseCodes(new int[] { 400 });
return br;
}
public void correctDownloadLink(DownloadLink link)
{
String url_source = link.getPluginPatternMatcher();
String urlNew;
if (link.getPluginPatternMatcher().matches("https?://(?:www\\.)?(?:nowtv|tvnow)\\.(?:de|ch)/[^/]+/[a-z0-9\\-]+/[^/\\?]+")) {
String url_part = new Regex(url_source, "https?://[^/]+/(.+)").getMatch(0);
String urlNew = "https://www.tvnow.de/" + url_part;
link.setLinkID(url_part);
}
else
{
String rubbish = new Regex(url_source, "(/(?:preview|player)(?:.+)?)").getMatch(0);
String urlNew; if (rubbish != null) {
urlNew = url_source.replace(rubbish, "");
} else {
urlNew = url_source;
}
rubbish = new Regex(urlNew, "(\\?.+)").getMatch(0);
if (rubbish != null) {
urlNew = url_source.replace(rubbish, "");
}
Regex sourceregex = new Regex(urlNew, "https?://[^/]+/([^/]+)/([a-z0-9\\-]+)");
String name_tvstation = sourceregex.getMatch(0);
String name_series = sourceregex.getMatch(1);
String name_episode = new Regex(urlNew, "/([^/]+)$").getMatch(0);
String url_part = name_series + "/" + name_episode;
urlNew = "https://www.tvnow.de/" + name_tvstation + "/" + url_part;
link.setLinkID(url_part);
}
link.setPluginPatternMatcher(urlNew);
}
public DownloadLink.AvailableStatus requestFileInformation(DownloadLink downloadLink)
throws Exception
{
setBrowserExclusive();
correctDownloadLink(downloadLink);
prepBR(br);
String urlpart = getURLPart(downloadLink);
br.getPage("https://api.tvnow.de/v3/movies/" + urlpart + "?fields=" + getFields());
if (br.getHttpConnection().getResponseCode() != 200) {
throw new PluginException(32);
}
entries = ((LinkedHashMap)JavaScriptEngineFactory.jsonToJavaObject(br.toString()));
LinkedHashMap<String, Object> format = (LinkedHashMap)entries.get("format");
String tv_station = (String)format.get("station");
String formatTitle = (String)format.get("title");
return parseInformation(downloadLink, entries, tv_station, formatTitle);
}
public static String getFields()
{
return "*,format,packages,isDrm";
}
public static DownloadLink.AvailableStatus parseInformation(DownloadLink downloadLink, LinkedHashMap<String, Object> entries, String tv_station, String formatTitle) {
MediathekProperties data = (MediathekProperties)downloadLink.bindData(MediathekProperties.class);
String date = (String)entries.get("broadcastStartDate");
String episode_str = new Regex(downloadLink.getPluginPatternMatcher(), "folge\\-(\\d+)").getMatch(0);
int season = (int)JavaScriptEngineFactory.toLong(entries.get("season"), -1L);
int episode = (int)JavaScriptEngineFactory.toLong(entries.get("episode"), -1L);
boolean isDRM = ((Boolean)entries.get("isDrm")).booleanValue();
if ((episode == -1) && (episode_str != null))
{
episode = (int)Long.parseLong(episode_str);
}
String description = (String)entries.get("articleLong");
String title = (String)entries.get("title");
if ((title == null) || (formatTitle == null) || (tv_station == null) || (date == null))
{
return DownloadLink.AvailableStatus.UNCHECKABLE;
}
String filename_beginning = "";
DownloadLink.AvailableStatus status;
DownloadLink.AvailableStatus status; if (isDRM) {
TvnowDe.TvnowConfigInterface cfg = (TvnowDe.TvnowConfigInterface)PluginJsonConfig.get(TvnowDe.TvnowConfigInterface.class);
filename_beginning = "[DRM]";
DownloadLink.AvailableStatus status; if (cfg.isEnableDRMOffline())
{
downloadLink.setAvailable(false);
status = DownloadLink.AvailableStatus.FALSE;
}
else {
downloadLink.setAvailable(true);
status = DownloadLink.AvailableStatus.TRUE;
}
}
else {
downloadLink.setAvailable(true);
status = DownloadLink.AvailableStatus.TRUE;
}
data.setShow(formatTitle);
data.setChannel(tv_station);
data.setReleaseDate(getDateMilliseconds(date));
if ((season != -1) && (episode != -1)) {
data.setSeasonNumber(season);
data.setEpisodeNumber(episode);
if (title.matches("Folge \\d+"))
{
title = null;
} else if (title.matches("Folge \\d+: .+"))
{
title = title.replaceAll("(Folge \\d+: )", "");
}
}
if (!StringUtils.isEmpty(title)) {
data.setTitle(title);
}
String filename = filename_beginning + MediathekHelper.getMediathekFilename(downloadLink, data, false, false);
try {
if (FilePackage.isDefaultFilePackage(downloadLink.getFilePackage())) {
FilePackage fp = FilePackage.getInstance();
fp.setName(formatTitle);
fp.add(downloadLink);
}
if ((!StringUtils.isEmpty(description)) && (downloadLink.getComment() == null)) {
downloadLink.setComment(description);
}
}
catch (Throwable e) {}
downloadLink.setFinalFileName(filename);
return status;
}
private void download(DownloadLink downloadLink) throws Exception
{
boolean isFree = ((Boolean)entries.get("free")).booleanValue();
boolean isDRM = ((Boolean)entries.get("isDrm")).booleanValue();
String movieID = Long.toString(JavaScriptEngineFactory.toLong(entries.get("id"), -1L));
if (movieID.equals("-1")) {
throw new PluginException(4194304);
}
if (isDRM)
{
throw new PluginException(131072, "Unsupported streaming type [DRM]");
}
String urlpart = getURLPart(downloadLink);
br.getPage("https://api.tvnow.de/v3/movies/" + urlpart + "?fields=manifest");
entries = ((LinkedHashMap)JavaScriptEngineFactory.jsonToJavaObject(br.toString()));
entries = ((LinkedHashMap)entries.get("manifest"));
String hdsMaster = (String)entries.get("hds");
String hlsMaster = (String)entries.get("hlsclear");
if (StringUtils.isEmpty(hlsMaster)) {
hlsMaster = (String)entries.get("hlsfairplay");
}
if (!StringUtils.isEmpty(hlsMaster)) {
hlsMaster = hlsMaster.replaceAll("(filter=.*?)(&|$)", "");
br.getPage(hlsMaster);
HlsContainer hlsbest = HlsContainer.findBestVideoByBandwidth(HlsContainer.getHlsQualities(br));
if (hlsbest == null)
{
throw new PluginException(131072, "Unsupported streaming type [DRM]");
}
checkFFmpeg(downloadLink, "Download a HLS Stream");
try {
dl = new HLSDownloader(downloadLink, br, hlsbest.getDownloadurl());
}
catch (Throwable e)
{
throw new PluginException(131072, "Unsupported streaming type [DRM]");
}
dl.startDownload();
}
else {
if (!isFree)
{
throw new PluginException(131072, "Download nicht möglich (muss gekauft werden)");
}
throw new PluginException(131072, "HDS streaming is not (yet) supported");
}
}
private String getURLPart(DownloadLink dl)
throws PluginException
{
Regex urlInfo = new Regex(dl.getPluginPatternMatcher(), "https?://[^/]+/[^/]+/([^/]*?)/([^/]+/)?(.+)");
String showname = dl.getStringProperty("url_showname", null);
String episodename = dl.getStringProperty("url_episodetitle", null);
if (StringUtils.isEmpty(showname)) {
showname = urlInfo.getMatch(0);
showname = cleanupShowTitle(showname);
}
if (StringUtils.isEmpty(episodename)) {
episodename = urlInfo.getMatch(2);
episodename = episodename.replaceAll("^episode\\-\\d+\\-", "");
if ((!episodename.matches(".*?(folge|teil|w)\\-\\d+$")) && (!episodename.matches(".+\\d{4}\\-\\d{2}-\\d{2}-\\d{2}-\\d{2}-\\d{2}$")) && (!episodename.matches(".+\\-[12]\\d{3}"))) {
episodename = episodename.replaceAll("\\-\\d+$", "");
}
}
if ((StringUtils.isEmpty(showname)) || (StringUtils.isEmpty(episodename))) {
throw new PluginException(4194304);
}
String urlpart = showname + "/" + episodename;
return urlpart;
}
public static String cleanupShowTitle(String showname)
{
if (showname == null) {
return null;
}
showname = showname.replaceAll("\\-\\d+$", "");
return showname;
}
public String getAGBLink()
{
return "http://rtl-now.rtl.de/nutzungsbedingungen";
}
public int getMaxSimultanFreeDownloadNum()
{
TvnowDe.TvnowConfigInterface cfg = (TvnowDe.TvnowConfigInterface)PluginJsonConfig.get(TvnowDe.TvnowConfigInterface.class);
if (cfg.isEnableUnlimitedSimultaneousDownloads()) {
return -1;
}
return 1;
}
public void handleFree(DownloadLink downloadLink)
throws Exception
{
requestFileInformation(downloadLink);
download(downloadLink);
}
public void reset() {}
public void resetDownloadlink(DownloadLink link)
{
if (link != null) {
link.removeProperty("RESUME_FRAGMENT");
}
}
public void resetPluginGlobals() {}
public static long getDateMilliseconds(String input)
{
if (input == null) {
return -1L;
}
return TimeFormatter.getMilliSeconds(input, "yyyy-MM-dd HH:mm:ss", Locale.GERMAN);
}
public Class<? extends PluginConfigInterface> getConfigInterface()
{
return TvnowDe.TvnowConfigInterface.class;
}
}1080p is not available.
python -m youtube_dl https://www.tvnow.de/serien/gzsz-1/2019-01/episode-6671-leon-ist-entsetzt-1354104 -F
[TVNow] gzsz/leon-ist-entsetzt: Downloading JSON metadata
[TVNow] 1354104: Downloading MPD manifest
[TVNow] 1354104: Downloading ISM manifest
[TVNow] 1354104: Downloading m3u8 information
[info] Available formats for 1354104:
format code extension resolution note
mss-audio-128 isma audio only 128k , AACL (48000Hz)
dash-audio=128000 m4a audio only [de] DASH audio 128k , m4a_dash container, mp4a.40.2 (48000Hz)
hls-136 mp4 audio only 136k , mp4a.40.2@128k
mss-video-500 ismv 640x360 500k , AVC1, video only
dash-video=500000 mp4 640x360 [en] DASH video 500k , mp4_dash container, avc1.4D001F, 25fps, video only
mss-video-1000 ismv 640x360 1000k , AVC1, video only
dash-video=1000000 mp4 640x360 [en] DASH video 1000k , mp4_dash container, avc1.4D001F, 25fps, video only
mss-video-1500 ismv 960x540 1500k , AVC1, video only
dash-video=1500000 mp4 960x540 [en] DASH video 1500k , mp4_dash container, avc1.4D001F, 25fps, video only
mss-video-3000 ismv 960x540 3000k , AVC1, video only
dash-video=3000000 mp4 960x540 [en] DASH video 3000k , mp4_dash container, avc1.4D001F, 25fps, video only
hls-666 mp4 640x360 666k , avc1.77.31@ 500k, 25.0fps, mp4a.40.2@128k
hls-1196 mp4 640x360 1196k , avc1.77.31@1000k, 25.0fps, mp4a.40.2@128k
hls-1726 mp4 960x540 1726k , avc1.77.31@1500k, 25.0fps, mp4a.40.2@128k
hls-3316 mp4 960x540 3316k , avc1.77.31@3000k, 25.0fps, mp4a.40.2@128k (best)1080p for example, over api url:
https://vodnowusodash.secure.footprint.net/proxy195/clear/ngvod/tvnow/1354104-1-15729.ism/.mpd?filter=(type%3d%3d%22audio%22)%7c%7c(type%3d%3d%22video%22%26%26systemBitrate%3c8616000)
krisrok
commented
5 years ago issue persists but is closed.
Please follow the guide below
xinto all the boxes [ ] relevant to your issue (like this:[x])Make sure you are using the latest version: run
youtube-dl --versionand ensure your version is 2018.12.09. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.Before submitting an issue make sure you have:
What is the purpose of your issue?
The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue
If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the
-vflag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):
Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
Description of your issue, suggested solution and other information
Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible. If work on your issue requires account credentials please provide them or explain how one can obtain them.