ZDF broken

[raptor2101]

ZDF does a complete overhaul of there frontend ...

[raptor2101]

this will take longer ^^

[raptor2101]

OK after a first look:

• direct access to the video-streams still possible
• zdf does some content hiding

because of the later i'am in contact with the ZDF - currently waiting for answers

[hnms]

Hi @raptor2101, I just noticed the ZDF bring broken and I'm very pleased to see that you're already working on it. Kudos! Do you operate a flattr account?

[raptor2101]

Thanks. No I don't run a flattr account (and I won't). Explanation one paragraph ahead.

If you are trying to crawl the new zdf website, you have to go this way: main site -> video page -> json data-> video config. The streaming links are offered in the last page. The json page is somehow "protected". My guess: the request for the json page has to be done within an already open connection plus some special request headers....

And here the fun starts: this technic could be installed as part of some flood/ddos protection or as some kind of copy protection. In case of the later one I will get in trouble if I bypass them. To avoid this, I start to contact the ZDF Webmaster yesterday... And now it comes very handy that I don't have some kind revenue for this plugin (nor any connection to kodi). I'm just a fan who want to support....

Long story short: because of the German local legal situation (even bypassing anti add blocker is suible) , I have to negotiate with someone to get clearance to crawl the site...

[hnms]

Okay, I see. Well, let's say we have come a long way, but "Neuland" it still is ;) When I read yesterday that the federal and regional governments are investigating how ad blockers could be outlawed, I was reminded of the US government trying to outlaw export of crypto software. Good luck with that, guys...

-------- Original Message -------- From: "Christian Kölpin" notifications@github.com Sent: October 29, 2016 9:32:36 AM GMT+02:00 To: raptor2101/Mediathek Mediathek@noreply.github.com Cc: hnms hnms@posteo.org, Comment comment@noreply.github.com Subject: Re: [raptor2101/Mediathek] ZDF broken (#85)

Thanks. No I don't run a flattr account (and I won't). Explanation one paragraph ahead.

If you are trying to crawl the new zdf website, you have to go this way: main site -> video page -> json data-> video config. The streaming links are offered in the last page. The json page is somehow "protected". My guess: the request for the json page has to be done within an already open connection plus some special request headers....

And here the fun starts: this technic could be installed as part of some flood/ddos protection or as some kind of copy protection. In case of the later one I will get in trouble if I bypass them. To avoid this, I start to contact the ZDF Webmaster yesterday... And now it comes very handy that I don't have some kind revenue for this plugin (nor any con Necton to kodi). I'm just a fan who want to support....

Long story short: because of the German local legal situation (even bypassing anti ad blocker is suible) , I have to negotiate with some someone to get clearance to crawl the site...

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/raptor2101/Mediathek/issues/85#issuecomment-257076840

[hnms]

I'll be in touch with ZDF as well to let them know I miss my Kodi plugin :)

-------- Original Message -------- From: "Christian Kölpin" notifications@github.com Sent: October 29, 2016 9:32:36 AM GMT+02:00 To: raptor2101/Mediathek Mediathek@noreply.github.com Cc: hnms hnms@posteo.org, Comment comment@noreply.github.com Subject: Re: [raptor2101/Mediathek] ZDF broken (#85)

Thanks. No I don't run a flattr account (and I won't). Explanation one paragraph ahead.

If you are trying to crawl the new zdf website, you have to go this way: main site -> video page -> json data-> video config. The streaming links are offered in the last page. The json page is somehow "protected". My guess: the request for the json page has to be done within an already open connection plus some special request headers....

And here the fun starts: this technic could be installed as part of some flood/ddos protection or as some kind of copy protection. In case of the later one I will get in trouble if I bypass them. To avoid this, I start to contact the ZDF Webmaster yesterday... And now it comes very handy that I don't have some kind revenue for this plugin (nor any con Necton to kodi). I'm just a fan who want to support....

Long story short: because of the German local legal situation (even bypassing anti ad blocker is suible) , I have to negotiate with some someone to get clearance to crawl the site...

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/raptor2101/Mediathek/issues/85#issuecomment-257076840

[raptor2101]

hopefully the listen, because that is on of their "highlightes": "available on all platforms" ...

let me dream of an direct Access (XML,JSON, whatever) for plugins to browse their whole cataloge :)

[stelas]

Hi @raptor2101, the 403 error seems to be a kind of token based HTTP authentication issue. This JSON file includes the right value named "apiToken". You need to add it to your HTTP request:

GET /content/documents/drama-im-ewigen-eis-die-verschollene-expedition-des-john-100.json?profile=player HTTP/1.1 Host: api.zdf.de Api-Auth: Bearer d2726b6c8c655e42b68b0db26131b15b22bd1a32

I've documented an working example here...

[raptor2101]

Thanks for that info, this saves me some testing time...

But I still waiting for an answer...

[jocki84]

Here are some hints that might be helpful, gleaned from the ZDF Mediathek 4.0 android app.

First, the URL zdf_mediathek_android_live_4_0.json provides a list of relevant URLs and URL templates, among others zdfCategoriesPage. That location provides a list of teasers for each of the categories, such as "Comedy", "Dokumentation", "Filme", "Gesellschaft" &c. Contained within are long descriptions along with the programme URLs (example). Those links, in turn, provide links for the various "formitaeten" such as this.

In the first file, there's also an API token generator zdfApiAuthToken, as well as a "search" URL template zdfSearch.

HTH

[raptor2101]

i'am currently working on a first JSON based version...

[raptor2101]

As you cas see, i just finished my first tries on using the "new" json-api. Works for the start page. Its fare from "usable".

Currently im thinking about, how to improve performance (do not load 150 JSONs at once to get direct links), how to proper handle these new "sub categories" (aka cluster) and how to fit this in the "old" underlying plugin structure (or to overhaul the whole thing, whitout breaking old things up ...)

[jocki84]

@raptor2101 It seems like we duplicated some effort… I got carried away and went ahead and built this version, my pull request #87 is based on your original version of zdf.py.

[raptor2101]

With my last commit(6fdd0c34af9ddc460b1e65649f35f2ae3648199e) browsing the ZDF Mediathel should be functional except LiveTV... i will look into this later.

but now I need testers, because I use new libs and I'am unaware of if any platform will provide them (especial sha256)

A little warning: the addon will create lot of json files, as you walks trough the mediathek, without deleting them, I will implement this later ;) ...

[esieke]

Awesome!

I did a short test (f89842f). OpenELEC 6.95.3 RPi.arm Only the menu "Sendung verpasst?" doesn't work. I think the url ist broken.

For information only: With an older version of OpenELEC on RPi i had a fault caused by the urllib2 library. The newest (OpenELEC Beta Releases (7.0 beta 3)) is recommended.

short test (2bda110) Live TV works :)

Thank you very much!

[raptor2101]

I'am currenty working on "Sendung verpasst", give me a few minutes ...

[raptor2101]

ok commit pushed ... try again

[esieke]

works for me! thanks

[raptor2101]

ok i wait till Monday for negative feedback. without negative feedback I will push these changes to kodi the kodi repo

[esieke]

what is your plan for ARTE?

[raptor2101]

Currently looking for it. See here https://github.com/raptor2101/Mediathek/pull/88 (i know, the discussion takes place at the wrong thread...)

[raptor2101]

Kurze antwort: es wird nicht so einfach wie bei zdf. Arte betreibt da ein bisschen vodoo...

[esieke]

Is this interesting? plugin.video.arteplussept

[ghost]

Hi, I have a problem with the current version (2352aa4). This is what I get when I try to open a menu: http://pastebin.com/raw/R1vpJ1mx

Maybe I did something wrong when installing the addon? I downloaded the master-zip from github and then in kodi installed it as addon from zipfile. I'm using kodi 16.0 on windows7.

[jocki84]

just a stab in the dark: are you able to find out and report your python version number?

It appears this could be a problem in python prior to 2.7.9. If you’re on an older version, any chance you could upgrade?

On 6 Nov 2016, at 22:23, kodi-user notifications@github.com wrote:

Hi, I have a problem with the current version (2352aa4 https://github.com/raptor2101/Mediathek/commit/2352aa4cc94222e40524e83064f66511717db4e6). This is what I get when I try to open a menu: http://pastebin.com/raw/R1vpJ1mx http://pastebin.com/raw/R1vpJ1mx Maybe I did something wrong when installing the addon? I downloaded the master-zip from github and then in kodi installed it as addon from zipfile. I'm using kodi 16.0 on windows7.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/raptor2101/Mediathek/issues/85#issuecomment-258711756, or mute the thread https://github.com/notifications/unsubscribe-auth/APV0Y8zgglvNXGMi_YVJHdjQPySf2x3iks5q7kVWgaJpZM4KjsFs.

[jocki84]

Looking at the source, it appears Kodi 16 (Jarvis) ships with Python 2.7.8 on Windows, while Kodi 17 (Krypton) will ship Python 2.7.11 as of 17.0a1.

I'll look into finding a workaround that works on python versions shipped with current Kodi on Windows.

[jocki84]

Did some investigating. The server requires TLSv1 with »Server Name Indication« (SNI). This can be seen by comparing the command

openssl s_client -connect zdf-cdn.live.cellular.de:443 -tls1

which fails to the command

openssl s_client -connect zdf-cdn.live.cellular.de:443 -tls1 -servername zdf-cdn.line.cellular.de

which succeeds.

Unfortunately, support for that extension was only added to Python 2.7.9, see PEP466.

So the upshot is that I don't see how this can be supported on Windows before Kodi 17 (Krypton) is released with an updated version of Python.

[raptor2101]

This can't be handled by the plugin, cause the TLS is completely coeverd by the used urllib. I see no usable way to bypass this.

So I have to set the dependency to Kodi 17 ...

[ghost]

I can confirm it works with Kodi 17.0 nightly.

[raptor2101]

Pull Request is out ...

[tonynieman]

I was waiting for an official addon-update for my kodi v16 under libreelec, but until now there is nothing available. Because python 2.7.11 is installed I thought the new addon-version should also work there too, without upgrading to kodi v17. So I first tried to update the addon, but fails with an error something like “Dependency not met … python 2.25.0”. Then I changed the version in the line “<import addon="xbmc.python" version="2.25.0"/> in the addon.xml to “… version="2.19.0". After that the addon updated and works fine on my raspberry.

Thanks for the update! Hopefully the described “workaround” helps some not so “advanced” kodi-users who also don’t want to update to v17.

Keep also in mind that you have to rename the zip and the first folder in the zip to “plugin.video.mediathek”

[raptor2101]

I know this problem but, i cannot lowering the dependency because of the Windows Version of Kodi wich ships it's own Python version ... this is currently creepy ...

theoreticaly you can run the plugin within a gotham kodi as long as you keep the pyhton version up to date ...

[Mangnoppa]

Hi, EDIT: "Wer lesen kann ist klar im Vorteil!" I did not read the last two comments. Sorry for that. installing from zip does not work. (downloaded master aca371a4815c9e2f42ff60972ba5df8dd761b270 and tried 'install from zip')

Here is the error-msg from ~/.kodi/temp/kodi.log: ERROR: CAddonInstallJob[plugin.video.mediathek]: Die Abhängigkeit auf xbmc.python in Version 2.25.0 konnte nicht aufgelöst werden.

Looks like the commit I checked out is not working on: LibreELEC (official) - Version: 7.0.2 (Jarvis) RPi3

Installing the addon manually works quiet well.

Thank you very much for your efford. This is one of the best addons ever!

[raptor2101]

@Mangnoppa: is written in my comment above - the Plugin needs the dependency xbmc.python 2.25.0 wich forces Kodi to Krypton. This is caused because only krypton guarantees that a Python 2.7.9 is available. Os long as you keep the Python version up to date by you self, you can run the plugin on gotham. But you have to install it manually...

sorry for that

[Siggi0904]

@raptor2101: Ich habe die aktuelle Version aus dem GIT unter LibreElec 7.0.2 auf der Wetek Play getestet. Sendungen, z.B. "ZDF - Die Anstalt" erzeugen Fehler beim Wiedergabeversuch und das Log enthält viele Mediathek-Meldungen (Debugs und Fehler)

Hier der Log-Auszug dem Mediathek-Plugin betreffend: http://pastebin.com/SaSe9hne

Angaben zur Wetek-Play: Starting Kodi (16.1 Git:c327c53). Platform: Linux ARM 32-bit Using Release Kodi x32 build Kodi compiled Jun 22 2016 by GCC 5.3.0 for Linux ARM 32-bit version 3.10.99 (199267) Running on LibreELEC (official) - Version: 7.0.2, kernel: Linux ARM 32-bit version 3.10.99 FFmpeg version: 2.8.6 Host CPU: ARMv7 Processor rev 0 (v7l), 2 cores available ARM Features: Neon enabled

[raptor2101]

Ich schau mir mal an was man da machen kann ;)

[raptor2101]

Du musst eine alte version verwenden die LineNumbers passen nicht. In der aktuellen wird dieser Fehler abgefangen ...

[raptor2101]

Since https://github.com/xbmc/repo-plugins/pull/756 is merged now, I Close this issue

[Siggi0904]

Dann hilf mir bitte, die aktuellen Dateien zu finden. Ich habe die Dateien von https://github.com/raptor2101/Mediathek/tree/pre-krypton verwendet. Wenn, wie ich lese, der branch einige Versionen hinter dem Master hängt, bitte ich um Aktualisierung.

[raptor2101]

der pre-krypton tree ist nur dafür gedacht, dass ich ein Lieferung in in des Kodi repository bekomme, der das plugin als "defekt" markiert.

im Master Zweig findest du die aktuellste version, die allerdings auf krypton als version aufsetzt.

[Siggi0904]

Siehst, und ich ich brauch eine Version für pre-Krypton. Die Version ist ja noch onLife, wäre gute, wenn du die aktualisieren könntest.

[raptor2101]

kurz und knapp, nein. Mir fehlt hier das Setup um gemeldete Fehler von X- Versionen nachzustellen. Vor allem wenn sie sich in so grundlegenden Funktionen wie TLS unterscheiden.

Du kannst du krypton version installieren, die dependency sind rein ein aktuelles Python ... Wenn das von Deiner Distribution nicht angeboten wird, wird du prinzipiell ein Problem mit den neuen Mediatheken haben ...

[Siggi0904]

Was ist denn das Problem bei der Pre-Krypton Umgebung? Zu altes Python, hab ich verstanden. Kann man das umgehen? Wo in den Quellen tritt dann das Problem auf?

Ich, wie viele andere auch, nutze die letzte stabile 7er LibreElec-Version (7.02)

[raptor2101]

Siehe oben.... einige Mediatheken (arte,zdf) setzten TLS mit SNI ein. Das wird erst ab Python 2.7.9 supported. Ergo hat das Plugin als requirement Python 2.7.9. Da man das in Kodi so nicht als Dependency angeben kann, habe ich, nach Rücksprache mit dem Kodi Team, die Dependency auf Krypton hochgezogen.

Nur so ist sichergestellt, das auf allen Plattformen (auch windows) ein passendes Python zur Verfügung steht.

[Siggi0904]

OK, Danke. Ich hab schon gesehen, in Jarvis ist das Addon als defekt markiert.