Integrate with Kodi PVR

[lovecrafthowardphillips]

Would it be possible to have a folder directly linking to all the channel live streams, cf the VRT Nu addon? This would allow for using this folder in widgets where the program info gets automatically updated as well. By "program info" I mean the information showing which program is currently being broadcast from when to when on that particular channel.

I can add all the direct links to the live streams of the different channels to my (super) favourites which is okay for starting the streams but it freezes the program info to the moment of creating the favourite. It would be great if I could immediately see from my widget which programs are being broadcast on the different channels like I can for the VRT Nu live streams.

Thanks!

[michaelarnauts]

Can't you use the channels menu for this? This also shows the currently airing program. I don't like to clutter the menu with another item that does almost the same as the channels menu.

The real issue I want to fix is to have Addons integrate with the Kodi TV Guide and Live TV. There are ways to do this, but I haven't had the chance to look into this. This is something VRT Nu could also do then, as it would greatly improve the experience to watch live TV trough Addons on Kodi.

[michaelarnauts]

Note to self: It's this add-on I'm thinking about, but this seems to be removed from Matt Huisman's repository.

https://www.matthuisman.nz/2019/02/iptv-merge-kodi-add-on.html

EDIT: A copy is available here, but I can't find a public git repository. http://k.slyguy.xyz/.repo/plugin.program.iptv.merge/

[michaelarnauts]

Another Note: Simple IPTV client also supports video-on-demand from the Guide now. This looks nice!

https://github.com/kodi-pvr/pvr.iptvsimple/blob/Matrix/README.md

[michaelarnauts]

Dream scattered: https://github.com/kodi-pvr/pvr.iptvsimple/issues/161

[lovecrafthowardphillips]

Can't you use the channels menu for this? This also shows the currently airing program. I don't like to clutter the menu with another item that does almost the same as the channels menu.

That would work of course but would also mean it's no longer one simple click to start the stream but two. No worries, though, I understand!

[dagwieers]

In VRT NU hebben we gekozen om de live TV kanalen ook nog eens apart aan te bieden, net zoals we ook de TV gids nog eens apart aanbieden. Dit omdat we denken dat het gebruik van mensen verschillend is en vanuit het hoofdmenu krijg je een goed overzicht van alle mogelijkheden.

Mensen denken niet altijd ik wil één bekijken, en dan live, maar eerder, wat is er nu op TV te bekijken live. Of wat komt er op TV vandaag of morgen, en dan kiezen ze het kanaal (zoals in een programmaboekje). Bovendien is het IMO ook beter om in het overzicht van de kanalen geen titels van live programma's te steken want als je die in je favorieten steekt (dit is de meest interessante link om toe te voegen), moet je altijd het live programma er weer uithalen.

Dus daarom doen we het bewust anders. IMO is het een voordeel dat er 2 manieren zijn om hetzelfde te bereiken. (bv TV gids via kanaal en dan tijdstip, of eerst het tijdstip en dan het kanaal)

[michaelarnauts]

Good news.

I've noticed that what has been written here (https://github.com/kodi-pvr/pvr.iptvsimple/issues/161) isn't correct. It works perfectly fine to use plugin:// urls in a playlist with the pvr.iptvsimple addon. I just never actually tried it.

When implemented, you can view the guide from within Kodi, change channels by pressing the number or ch+ and ch- on your remote. It doesn't matter from what addon the channels are, or if they are widevine protected. In Kodi Matrix, it should be even be possible to play video on demand items from the guide itself.

I'm thinking of a service (independent from plugin.video.vtm.go or plugin.video.vrt.nu) that receives channel info and guide info trough json messages (like upnext allows to communicate between addons), and this service will then have the responsibility to write the playlist and guide data in the right format on the right location for pvr.iptvsimple to pick this up.

#EXTM3U

#EXTINF:-1 tvg-logo="een.png",één
plugin://plugin.video.vrt.nu/play/id/vualto_een_geo

#EXTINF:-1 tvg-logo="canvas.png",Canvas
plugin://plugin.video.vrt.nu/play/id/vualto_canvas_geo

#EXTINF:-1 tvg-logo="ketnet.png",Ketnet
plugin://plugin.video.vrt.nu/play/id/vualto_ketnet_geo

#EXTINF:-1 tvg-logo="vtm.png",vtm
plugin://plugin.video.vtm.go/play/catalog/channels/d8659669-b964-414c-aa9c-e31d8d15696b

[lovecrafthowardphillips]

Wow, that would be an extremely elegant solution. Looking forward to trying it out ;)

Van: "Michaël Arnauts" notifications@github.com Aan: "add-ons/plugin.video.vtm.go" plugin.video.vtm.go@noreply.github.com Cc: "dennis" dennis.van.tilborg@telenet.be, "Author" author@noreply.github.com Verzonden: Dinsdag 21 april 2020 10:35:46 Onderwerp: Re: [add-ons/plugin.video.vtm.go] Group all live streams under one folder (#168)

Good news.

I've noticed that what has been written here ( [ https://github.com/kodi-pvr/pvr.iptvsimple/issues/161 | kodi-pvr/pvr.iptvsimple#161 ] ) isn't correct. It works perfectly fine to use plugin:// urls in a playlist with the pvr.iptvsimple addon. I just never actually tried it.

When implemented, you can view the guide from within Kodi, change channels by pressing the number or ch+ and ch- on your remote. It doesn't matter from what addon the channels are, or if they are widevine protected. In Kodi Matrix, it should be even be possible to play video on demand items from the guide itself.

I'm thinking of a service (independent from plugin.video.vtm.go or plugin.video.vrt.nu) that receives channel info and guide info trough json messages (like upnext allows to communicate between addons), and this service will then have the responsibility to write the playlist and guide data in the right format on the right location for pvr.iptvsimple to pick this up.

[ https://user-images.githubusercontent.com/1193779/79844123-8a432280-83bb-11ea-91a2-36a540b847bd.png ]

EXTM3U

EXTINF:-1 tvg-logo="een.png",één

plugin://plugin.video.vrt.nu/play/id/vualto_een_geo

EXTINF:-1 tvg-logo="canvas.png",Canvas

plugin://plugin.video.vrt.nu/play/id/vualto_canvas_geo

EXTINF:-1 tvg-logo="ketnet.png",Ketnet

plugin://plugin.video.vrt.nu/play/id/vualto_ketnet_geo

EXTINF:-1 tvg-logo="vtm.png",vtm

plugin://plugin.video.vtm.go/play/catalog/channels/d8659669-b964-414c-aa9c-e31d8d15696b

— You are receiving this because you authored the thread. Reply to this email directly, [ https://github.com/add-ons/plugin.video.vtm.go/issues/168#issuecomment-617037885 | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AJGQSR3QQERCXIRJK35PSGTRNVLGFANCNFSM4LUSTCKA | unsubscribe ] .

[dagwieers]

I'm thinking of a service (independent from plugin.video.vtm.go or plugin.video.vrt.nu) that receives channel info and guide info trough json messages (like upnext allows to communicate between addons), and this service will then have the responsibility to write the playlist and guide data in the right format on the right location for pvr.iptvsimple to pick this up.

We discussed this before and I think that this would be the right approach. IMO such a service should:

• Allow for registering channels
• Allow for unregistering channels
• Automatically remove channels for add-ons that are not enabled/installed
• Detect changes from outside the service
• Properly lock and unlock the file to avoid corruption

We should probably look for other add-ons reading/writing to the configuration, or discuss our plans with the iptvsimple project to discuss alternative ways of doing this without using the existing file. (To avoid conflicts)

Ideally the iptvsimple add-on would ship with this service so it becomes the official interface to make updates, but I am not even sure you can mix a binary add-on with a python service.

[michaelarnauts]

I've temporary used EPG data from somewhere else, to give a sense of how this will work:

There is also plugin.video.regiotv from @dagwieers that has additional live streams. The only channels that currently have no available live stream are vier, vijf and zes.

[michaelarnauts]

I'm thinking of a service (independent from plugin.video.vtm.go or plugin.video.vrt.nu) that receives channel info and guide info trough json messages (like upnext allows to communicate between addons), and this service will then have the responsibility to write the playlist and guide data in the right format on the right location for pvr.iptvsimple to pick this up.

We discussed this before and I think that this would be the right approach. IMO such a service should:

• Allow for registering channels
• Allow for unregistering channels
• Automatically remove channels for add-ons that are not enabled/installed
• Detect changes from outside the service
• Properly lock and unlock the file to avoid corruption

We should probably look for other add-ons reading/writing to the configuration, or discuss our plans with the iptvsimple project to discuss alternative ways of doing this without using the existing file. (To avoid conflicts)

Ideally the iptvsimple add-on would ship with this service so it becomes the official interface to make updates, but I am not even sure you can mix a binary add-on with a python service.

I've seen issues in pvr.simpleiptv about this, and they were not interested in having such a service, or merging data from multiple sources. The closest thing that comes to mind is the plugin.program.iptv.merge plugin from @matthuisman but with no public repo, it's hard to work together.

[dagwieers]

Agreed. But there is potential that things may conflict. Another option is, rather than a service, we have a library for assisting add-ons to implement this. This could be part of kodiutils and could be made safe from a technical perspective, but there's always a conflict to how/where others add their channels and channels e.g. switching position, or being replaced.

At least if an add-on manages it, it could be considered authoritative for its own channels. Cleaning up channels would be a problem in this case, still.

This relates to add-ons/plugin.video.vrt.nu#209

[michaelarnauts]

Managing the channels is something that could be done from within the UI of Kodi itself, you can just hide the channels you don't want to see, change the numbers, change the name, ...

From the addon perspective, it should just broadcast the data it has. If an addon doesn't have a channel anymore (like VTM Kids Jr that got dropped), it would not be in the broadcast anymore, and the service can notice this and remove the channel.

Something like:

{
  "channels": [
    {
      "id": "een.be",
      "name": "één",
      "logo": "http://whatever/een.png or /home/user/.kodi/addons/plugin.video.vrt.nu/logos/een.png"
    },
    {
      "id": "canvas.be",
      "name": "canvas",
      "logo": "http://whatever/canvas.png or /home/user/.kodi/addons/plugin.video.vrt.nu/logos/canvas.png"
    }
  ]
}

And an additional broadcast for guide data:

{
  "channel": "een.be",
  "date": "2020-04-21",
  "guide": [
    {
      "begin": "2020-04-21 07:00:00",
      "end": "2020-04-21 07:30:00",
      "title": "Dagelijkse kost",
      "description": "Jeroen kookt blablabla",
      "image": "https://www.vrt.be/images/blablablablablaa-1234569999955444.png"
    },
    {
      "begin": "2020-04-21 07:30:00",
      "end": "2020-04-21 08:00:00",
      "title": "Het journaal",
      "description": "Nieuwsmagazine blablabla",
      "image": "https://www.vrt.be/images/blablablablablaa-1234569999955444.png"
    }
  ]
}

[michaelarnauts]

I've popped the question for more info of plugin.program.iptv.merge at https://forum.kodi.tv/showthread.php?tid=340691&pid=2942834#pid2942834

[michaelarnauts]

Thanks, Matt

The only real downside about IPTV Merge is that it's pretty closed (no repo), and depends on script.module.slyguy for example, that is not available in the Kodi repository. I would prefer to find something that can live in the official repositories.

The reason why I was thinking about those push messages, is because I don't have a xmltv guide for download. My addon needs to do multiple API calls to the service, and create and xmltv file itself, so I would need to do all those requests again (of the past days, and the next days) every time iptv merge does a pull, instead of when my addon does this.

[dagwieers]

BTW Support for radio channels would be needed as well. And maybe the user should be able to select which (radio/tv) channels they would like to expose this way.

[matthuisman]

What happens when addon is writing to file and service tries to read it

[michaelarnauts]

I have currently implemented it by writing to a .tmp file first, and then moving it to the destination. This is an atomic operation.

[matthuisman]

So addon writes to temp file then moves when done? I think i did this at first but then thought, what happens if service tries to read half way through that moving. Is that the reason for your sleep after checking for file?

[michaelarnauts]

The sleep is to not hammer the checking. As soon as the file has content, it is complete. You can't check the file halfway during a move. It's there or it's not.

[matthuisman]

Oh. So write to temp file in same directory. Then do a rename. I see that would be safe.

[michaelarnauts]

The current implementation is just a proof of concept. It is probably not allowed by the Kodi rules to read files outside your own addon directory.

The end goal is to have something that is approved by Team Kodi.

[dagwieers]

And an interface that doesn't require the add-on to implement a service, and is fairly easy to use.

[michaelarnauts]

I understand your view about this, but the goal was not to have an m3u or an xmltv merger. I prefer to hide the implementation details (m3u and xmltv) from the addons. Maybe there even is another PVR addon in the future that doesn't work with m3u and xmltv files.

I'm also looking at another way to pass the data from the add-on to the service, and the json files are smaller then the xmltv files.

[dagwieers]

It is not impossible to use a JSON format that maps exactly to xmltv, you just have to use a good convention that converts in both directions without data-loss. See http://wiki.open311.org/JSON_and_XML_Conversion/

A python implementation is here: https://github.com/sanand0/xmljson

[dagwieers]

Add-ons that get the EPG information from the VOD service do not have it as XMLTV. So it makes more sense they can create a nice dictionary and leave the XMLTV standard up to iptv.manager. So I think it makes sense to support both to keep it as clean and simple as possible for add-ons.

[phunkyfish]

I've been working with @matthuisman a lot recently to make sure iptvmerge works well with IPTV Simple client (at least in Matrix) I've also been asking him for something in a public repo as I feel a lot of users require this functionality but a non public repo is just not accessaible to many

If there are other things you would like to incorporate let me know, happy to contribute.Lets just try and avoid re-inventing the wheel where we can ;)

[michaelarnauts]

@phunkyfish I would like to have a solution that is allowed in the Kodi repository, so I will probably have to get rid of all the automatic configuration that I do now inpvr.iptvsimple, and provide documentation so the user can do this himself.

I'm not looking to replace IPTV Merge, although the use cases are really similar, but without an open repository, nor the intention of submitting it to the Kodi repository, IPTV Merge is not really an option for me. I understand that's something that slyguy has to do now, but since none of his addons have an open repository, I don't think he is interested in doing this.

So I'm not sure if I want to implement everything IPTV Merge does. Like @matthuisman said, he followed the path to make it as easy as possible, but this way requires reading in other addons paths, or changing other addons settings, and these things are not allowed by team kodi in the official repository.

My use case is for other addons to publish their channels and epg data to the Kodi PVR, and I don't want these addons to have to write xmltv files or m3u playlists, but that they can just use an API to register their stuff.

I could narrow service.iptv.manager down to just write one playlist.m3u and one epg.xml, and if the user wants to combine this with other sources, he can also use IPTV merge to merge this with his other sources.

[phunkyfish]

Are you on kodi slack @michaelarnauts ?

If not feel free to PM me your email (on the kodi forum) and I can get you added. It would allow the conversation to move a bit quicker.

[michaelarnauts]

Are you on kodi slack @michaelarnauts ?

If not feel free to PM me your email (on the kodi forum) and I can get you added. It would allow the conversation to move a bit quicker.

@phunkyfish I've tried to send you a PM, but I can't find out how, maybe I don't have enough posts yet, I'm not really active on the Kodi forum. You can find my email on my GitHub profile.

[phunkyfish]

No worries. Got it. You should get an invite some time today.

[phunkyfish]

Invite sent

[dagwieers]

I have written down some of the design decisions (or possible design changes). Please read this and provide feedback: https://github.com/add-ons/service.iptv.manager/wiki/Design

It is clear that Kodi limitations (and kodidev rules for add-ons) do not make it easy for us to find an acceptable mechanism to advertise and communication information between add-ons.

Update: While trying to make my point for using JSON-based XMLTV it is clear that Badgerfish or Abdera or really clunky mechanisms. I think we are better off with a more simple interface for add-ons. Most VOD services have their data in their own format, and it shouldn't be up to the add-on to translate this into XMLTV if the service can do it for them.

[phunkyfish]

Suggestion:

Create a backend for simpleiptv. This backend can either receive data on a rest endpoint from an addon or can query a rest endpoint on an addon for data (covers addons with/without a service). The backend can support whatever formats are required.

Then the backend can aggregate in whatever way required and expose an M3U and XMLTV endpoint to iptvsimple.

Once there is a endpoint to read from I can figure out how to cleanly read and update on Leia version similar to Matrix (or find a way to shoehorn it in!).

For addons with a service would even be possible to have a generic module with auto discovery I guess.

[dagwieers]

@phunkyfish What do you mean with a REST endpoint on an addon? Because I don't see how an add-on can have a REST endpoint without a (HTTP) service. Maybe there is something I have not heard of.

[phunkyfish]

What I meant was a simple addon without a service would just call a REST endpoint on on the backend. A more complex addon that has a service could expose a REST endpoint that the backend could read from.

Could even call it IPTV Simple Backend.

[phunkyfish]

Overtime the backend could be updated to support any format required (XML, JSON, M3U, protobufs) and connection type required (REST, file lookup, gRPC, jsonRPC etc.)

Using REST is nice as you can develop across the Leia/Matrix boundary easily. For instance build the backend on Leia, but against Matrix simpleIPTV client. Once that works port or Leia simpleIPTV client.

[dagwieers]

@phunkyfish I assume with a REST endpoint, you mean the service implements an HTTP service.

My preference would be to have a single mechanism for all add-ons. Which from that Wiki link technically boils down to:

• service → add-on
  • RunPlugin
  • RunScript
• add-on → service
  • Files
  • HTTP

Which means IPTV Manager would need to implement an HTTP service to accept data, because we have no other choice...

IMO it would make more sense that RunPlugin() and RunScript() could send (e.g. stdin) and return information (e.g. stdout and stderr). That would allow for a REST-like mechanism using plugin:// URLs which would be much easier to implement by add-ons in general.

I wouldn't rule out support for multiple formats, but to keep things simple for VOD add-ons, XMLTV or M3U8 support is not a primary concern IMO. (Usually VOD providers do not expose EPG data in XMLTV format). It is more efficient to leave this up to IPTV Manager to use XMLTV and M3U8 to feed to PVR IPTV Simple.Client.

[phunkyfish]

Cool, makes sense. If you use plugin:://URLs can they work across kodi instances?

That was my thinking was this could be configured to work across devices (different IPs) and Kodi versions (i.e. 18/19). But that may only work with HTTP. I'm not familiar with plugin:// URLs

[dagwieers]

@phunkyfish It is clear from Team Kodi that our implementation of IPTV Manager as was designed, will not be accepted for inclusion in the official Kodi repository. So we cannot have add-ons advertise supporting IPTV Manager through files, and we cannot have add-ons write XMLTV/M3U8 data for IPTV Manager to read back. Those would violate rules and an exception for IPTV Manager is not possible.

So we do not have many options at this point if we do not want to impose add-ons to implement a service.

What is more dramatic, is that we are also not allowed to change IPTV Simple settings or reload IPTV Simple (not even with explicit consent from the user, as we do now). So that makes integrating with IPTV Simple (as it works today) impossible. And for that matter, IPTV Merge will not be accepted as an official solution either.

What we would need from IPTV Simple (to even have something that would work out-of-the-box) is:

• that it automatically reloads its data when one or more files have been updated (e.g. by IPTV Manager)
• an API to add our channel listing and EPG information (without replacing existing configuration), this would mean IPTV Simple would have to support merging XMLTV and M3U files itself

Other design considerations have been updated in: https://github.com/add-ons/service.iptv.manager/wiki/Design

One such change is the use of sockets rather than file-based or HTTP-based communication for transfering channel and EPG information back from the add-on the IPTV Manager.

[phunkyfish]

As I said before the right way to do this is via HTTP as detailed in your Wiki page. It allows flexibility across kodi versions and platforms and it would be required for iptv simple to read from anyway.

Then each addon simply sends its data to iptv manager where it is aggregated. Note that doing the aggregation in a PVR addon is IMHO just wrong. That is the job of a backend. Which is also I suggested this be built as a backend for IPTV Simple 😉

There is already a facility to read data on a timed basis from iptvsimple. Implementing an async epg update should also be possible as a phase two.

There is a great opportunity here to not only solve this for video addons but also for users who required combining multiple playlists.

A solution like this could be the official supported solution and efficiencies built into iptvsimple to use it. But it would need to work for everyone.

[michaelarnauts]

If the backend is just a Kodi add-on with a webservice, aren't there better ways to communicate then HTTP? I think it's a bit strange that two components in one system have to communicate trough HTTP in my opinion. I was thinking about an xbmcprv module (like we have xbmc, xbmcvfs, xmcaddon, ...) that python Addons could use, and this could provide an API to register channels and epg, but I think that this would mean that it has to be a core Kodi feature.

Another issue with pushing data vs polling (what we do now) is that there is no precise moment in time where we know when all playlists are received. Every addon can then send its playlist or epg data when it wants and we don't know when we can start to merge. The only solution is to do a merge at every push that comes in.

This would then require caching all pushes to file, since we have to merge at every request that comes in, and we also need to merge previous data again.

For Kodi installations that run of a SD card, Disk IO is something you want to avoid as much as possible. This is the reason I think a polling update scheme is more appropriate.

Don't get me wrong, I want to do this the right way, but the suggestion from Team Kodi was to fork IPTV Simple and make the changes there. I don't think this is the solution that's best for Kodi or IPTV Simple.

[phunkyfish]

The communication method is less important here (but you would need HTTP for iptvsimple to read). The benefit you get from using kodi is that you can depend on the right things being installed for everything to work. If you want this to work across kodi versions you can’t depend on a core feature. gRPC would allow you to do what you want but that would mean building that into simple addons. Trade offs.

I agree with you that forking IPTV Simple is the wrong thing to do, that was not a good suggestion. Remember that PVR works today for many backends so we should not need any custom features to achieve this initially. Also remember that binary addons do not have the same access to the kodi API that python addons do so they tend to use a common comms method like http.

I think efficiencies around the EPG data, in memory processing etc. is less complex than you might think for a backend. It would only get complex when you require persistence.

[michaelarnauts]

We have something that works and should be valid according to the Addon rules. We should now test this, and see where we can improve.

See https://github.com/add-ons/service.iptv.manager

[phunkyfish]

You guys don't waste any time ;)

On Wed, 6 May 2020 at 17:36, Michaël Arnauts notifications@github.com wrote:

We have something that works and should be valid according to the Addon rules. We should now test this, and see where we can improve.

See https://github.com/add-ons/service.iptv.manager

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/add-ons/plugin.video.vtm.go/issues/168#issuecomment-624756409, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDDKYYUCIKMDOAYLM5R3ELRQGGX7ANCNFSM4LUSTCKA .

[dagwieers]

For completeness, the IPTV Manager add-on is now an official Kodi add-on. So a lot of what was discussed here has been resolved and was accepted. We still have a wish to use HTTP communication over the existing socket communication, but that is a technical detail which we can improve on in a subsequent release and would not require a big change to add-ons to get implemented when ready.

A big thanks to @michaelarnauts for doing most of the groundwork.