Matching using tvg-id instead of tvg-name

yuljk commented 5 years ago

Hi guys - Is there any way HLS-Proxy could be setup to match M3U playlists to an EPG using the tvg-id tag instead of tvg-name.

The reason for this is that I like to create my own M3U's from my IPTV provider and prefer to rename the channels (as the ones from my provider are prefixed with the country name etc)

Many thanks

warren-bank commented 5 years ago

Hi. Please let me make sure that I understand what you're trying to accomplish. If any of the following is incorrect, then please tell me how-so..

First, an M3U file is the the same format as an M3U8.. except that its character encoding isn't necessarily UTF8.

Using this snippet of M3U as an example:

#EXTM3U
...
#EXTINF:-1 tvg-ID="SRF1.ch" tvg-name="SRF 1 HD" tvg-logo="https://lut.im/LXJpuDO5hK/OM3ZjtxQt9O21hAo.png" group-title="DE",SRF 1 HD
http://xxxxxxxxx.xxxxx.net:8000/live/xxxxxx/53s7Fedzad/xxxx.ts
#EXTINF:-1 tvg-ID="SRF2.ch" tvg-name="SRF 2 HD" tvg-logo="https://lut.im/UyNtCWxYqt/2oDTna1FgStj17Yz.png" group-title="DE",SRF 2 HD
http://xxxxxxxxx.xxxxx.net:8000/live/xxxxxx/53s7Fedzad/xxxx.ts
#EXTINF:-1 tvg-ID="SRFinfo.ch" tvg-name="SRF Info HD" tvg-logo="https://lut.im/1m7YZq1iDS/StzZy97pEMpTvEHZ.png" group-title="DE",SRF Info HD
http://xxxxxxxxx.xxxxx.net:8000/live/xxxxxx/53s7Fedzad/xxxx.ts
...

You would like the proxy to pass this text through a filter function that applies conditional logic to modify the value of tvg-name attributes?

yuljk commented 5 years ago

Hi Warren - Many thanks for the swift response. Essentially I'm after the same functionality that is used by Kodi's Simple PVR Client as it allows matching of the tvg-id tag with the EPG instead of using the tvg-name tag.

Currently If I use my customised m3u file with Helix's EPG I see the following in HLS Proxy:-

https://imgur.com/ziCxc8S

You would like the proxy to pass this text through a filter function that applies conditional logic to modify the value of tvg-name attributes?

I'm not sure if what you're describing here would address the problem.

Kind Regards

warren-bank commented 5 years ago

I'll have to plead ignorance about what Simple PVR does. Could you please explain to me.. either what the proxy is breaking and you'd like fixed.. or what modification you'd like made to the M3U file's content as it passes through the proxy?

warren-bank commented 5 years ago

..to be upfront, this isn't a use-case I've tested; nevertheless, I don't see any reason it should be a problem.

yuljk commented 5 years ago

How does HLS Proxy currently match an M3U with an EPG? Which tags does it reference when matching?

warren-bank commented 5 years ago

it doesn't..

the parser is super simple.. it doesn't parse it as XML and inspect tags or attributes.. it simply uses a regular expression to identify all video links (ex: .ts files) within the m3u8.. and modifies them such that when the viewer requests those videos.. they are requested via the proxy, rather than directly from the server.

Depending on command-line configuration options, the proxy will pre-request these video files and store them in a local cache.. so when the viewer does request them from the proxy, then they're already available.

Currently, there is no hook to allow the user to modify the raw m3u8 content.. there is a hook to modify the URLs that are extracted from the m3u8.. if adding such a (raw) hook would be helpful, then it shouldn't be too difficult.. but I'm not sure if that would help you to solve your issue.

A complete example would be helpful...

an unmodified M3U that doesn't work as you want
a hand-modified M3U that looks exactly the way you need

Are you trying to change the value of each tvg-id attribute to match its corresponding tvg-name?

warren-bank commented 5 years ago

..since the proxy doesn't modify either tvg-id or tvg-name, I would assume that.. the proxied M3U should work the same in Kodi's Simple PVR Client as the unproxied M3U.. is that the case?

yuljk commented 5 years ago

I'll give it a test now.

warren-bank commented 5 years ago

to reduce the number of variables.. you should probably run your test without --prefetch

..if that works, then try again with it

yuljk commented 5 years ago

Tested with a couple of my IPTV clients and the EPG is blank on both.

yuljk commented 5 years ago

I've uploaded my playlist with custom channel names (which doesn't work with HLS Proxy) and the unmodified playlist provided by Helix IPTV. (Which Does)

yuljk commented 5 years ago

https://drive.google.com/file/d/1GPvYSUIpTGCaaP7am0B1928JzwuDNsdj/view?usp=sharing https://drive.google.com/file/d/1JRfDTzv-tTnzJaY7QaLPRwBBawz-p8PL/view?usp=sharing

warren-bank commented 5 years ago

ok.. what client should I use to try to view the playlist?

yuljk commented 5 years ago

I'm using Simple PVR Client for Kodi and IPTV Extreme on Android. The EPG doesn't display properly in HLS Proxy also - if you see the screenshot from my previous post

yuljk commented 5 years ago

If I bypass the proxy altogether and just point directly to my modified playlist the EPG loads fine.

warren-bank commented 5 years ago

well.. a few quick observations:

the .m3u file extension won't be recognized as an .m3u8
- so the manifest won't be processed.. and links within it won't be redirected through the proxy
even if the .m3u was renamed to .m3u8
- the video URLs within it don't end with a .ts file extension.. so they won't be cached when --prefetch is enabled

I'll run a few tests.. and see what I see. Will report back shortly..

warren-bank commented 5 years ago

whoa.. what format are these videos being served?

I just chose the first video URL and entered it into the browser URL bar: http://jammy.today:83/rVP3JEc3/C8xD2Ge0/330424

..and the download didn't stop until I forced it to stop ..a normal .m3u8 contains many small .ts chunks ..this thing is enormous

Is that a progressive .mp4 or... ?

yuljk commented 5 years ago

They are just provided by my IPTV provider in M3U plus format..

yuljk commented 5 years ago

I thought that was the standard format for IPTV streams?!

warren-bank commented 5 years ago

honestly.. I'd need to google for more info.

I used wget and downloaded about 20MB of this video.. and watched in VLC. It appears to be part of a normal .mp4 (h264) video file. If the download was allowed to continue longer.. I'd have the entire movie in a single .mp4 file. That's definitely different from HLS streaming.. but I'm not sure (offhand) what format it is.

yuljk commented 5 years ago

I was under the impression HLS Proxy would work with most IPTV providers. :(

warren-bank commented 5 years ago

well.. I wrote it with the intention of proxying HLS streams. Maybe it could be expanded to other formats, but I'd need to learn a bit more about them before saying one way or the other.

yuljk commented 5 years ago

Ahh - I've just checked their control panel and it looks like you can generate a playlist in m3u8 format. Although it generates a simple list without much in the way of tagging etc

yuljk commented 5 years ago

I'll get in touch with Helix support and see if I can get something in m3u8 format

warren-bank commented 5 years ago

well.. I will say this.. your streams appears to be very fast and stable; I'm not really sure if you'd benefit by using this proxy.

not to knock my own work.. but this project isn't perfectly polished.. and I mainly just use it as a work-around to deal with some of the harsher realities of video streams that exist in the wild: unstable, slow, restricted based on HTTP headers, etc..

warren-bank commented 5 years ago

..my point is, even I don't use it unless I have to

yuljk commented 5 years ago

I'd heard a few folk suggesting it was a good idea for those periods when extensive buffering occurs with various providers. Touch wood that hasn't been a regular occurrence with Helix - but during peak times certain sports channels do suffer.

yuljk commented 5 years ago

Cheers for your assistance Warren - much appreciated. I may hold off on HLS for the time being, unless I see more buffering issues/and/or they can spit out a M3U8 for me :)

warren-bank commented 5 years ago

I make you no promises.. but I will look into IPTV formats and try to figure out what they're serving.. and whether there's anything that a proxy could do to help.

Offhand.. since it appears to be just one continuous data stream, I can't think of anything a proxy could do (as a man-in-the-middle) to improve the quality of the stream at the final receiver / video player.

warren-bank commented 5 years ago

cheers.. sorry I wasn't more help

yuljk commented 5 years ago

Hi Warren - I managed to get HLS Proxy working correctly once my IPTV provider issued me with a proper m3u8 playlist. Success!

The one thing I have noticed is that the characters in my EPG are in Russian. If I navigate to hlsproxyaddress/epg and download the epg and examine it in notepad all entries are as follows:-

<channel id="hlsproxy-4"><display-name lang="ru">11 канал (Днепр)</display-name></channel>
<channel id="hlsproxy-5"><display-name lang="ru">112 Украина</display-name></channel>

I should note that the source playlist is all in English, so it seems to relate to language settings in HLS Proxy.

Could this be changed over to English?

Thanks

evoactivity commented 4 years ago

@yuljk I think your looking for https://hls-proxy.com/ which is designed for IPTV playlists

warren-bank / HLS-Proxy

Matching using tvg-id instead of tvg-name #2