Handling reauth of a channel m3u that redirects to a CDN back-end?

[bruor]

You seem to be open to exploring new use cases, how about this one? I'm currently using https://www.hls-proxy.com/ and looking for an alternative since it can't handle this.

I've been doing some testing and noticed that my provider requires regular re-auth of the stream to keep it alive.

The channel M3U URL looks like this inside the master playlist: https://tvnow.best/api/stream/username/password/livetv.epg/channel.name.m3u8

When you GET it you receive a 301 redirect to something like this which is hosted on cloudflare: https://183-81-168-251.servers.party:5052/live22/DnUJ1vfuNxq-KIzI2GFbvw/312330/1706784248/3041567.m3u8

After an amount of time, the cloudflare servers throw a 410 error or 403 error when you try to GET the M3U or the .ts files and the stream drops. If you re-call the channel level m3u you are happily redirected to the content and can resume playing it.

Any chance you would be interested in adding this type of error handling / channel source resume functionality into your proxy?

[warren-bank]

Interesting problem.. it would only effect live streams (not vod)..

off-hand, I'd guess that this is probably video player dependent; whereby one player would re-request the original manifest URL, and another player would re-request the URL that is the result of redirection.

If so, then the former player would work fine with your provider, and the latter would not (as its auth token would eventually expire).

Also, if so, then the question you pose is.. could this proxy be used to help to workaround this issue when used in combination with the latter player?

Currently, no. However, I could imagine how a workaround could be implemented with the addition of two new hook functions:

1. onManifestRedirect(urlsArray)
2. onManifestError(url, status)

for example (untested, just some quick pseudo-code):

// hooks.js

// map: lastURL to firstURL
const manifestsMap = {}

const onManifestRedirect = (urlsArray) => {
  if (urlsArray && Array.isArray(urlsArray) && (urlsArray.length > 1)) {
    const lastURL = urlsArray[urlsArray.length - 1]
    const firstURL =urlsArray[0]
    manifestsMap[lastURL] = firstURL
  }
}

const onManifestError = (url, status) => {
  if ((status >= 400) && (status < 500) && manifestsMap[url]) {
    return {
      "status": 301,
      "url": manifestsMap[url]
    }
  }
}

module.exports = {
  onManifestRedirect,
  onManifestError
}

[bruor]

I don't think I've come across a player that works around it yet, though I haven't had the time to test to that degree. The streams can take 4h to cut out sometimes and I don't have enough free time available to try to repro in something like VLC. I can pull examples from past logging though if you want to see exactly what happens when the other hls-proxy errors out.

I'm still learning HLS terminology, manifest would be the m3u that lists the .ts segments? If so I'm not completely sure if the error is thrown when it's trying to fetch the manifest or only the resulting .ts segments but I can pull some examples from my logs to verify.

If I'm understanding this code properly you're essentially keeping track of the URL you called vs the URL you ended up at to get the m3u, then if you receive an error you are redirecting yourself back to the 1st URL so you can be redirected again.

How would I go about coding/testing a hook function? Assuming the above was functional would the proxy understand what to do with those new functions? I'm seasoned when it comes to infrastructure but green when it comes to code.

[warren-bank]

In so far as your understanding of what that code is doing.. you're absolutely correct. However, you can't actually test that code yet.. because the hook functions that I'm proposing aren't yet supported by the proxy; I'd need to add code in the proxy to call them at the appropriate times. The proxy already supports a fairly long list of hook functions.. so it's fairly trivial to add support for additional ones.

HLS terminology is pretty simple:

• .m3u8 files are manifests.. which are essentially just playlists
  • a "master" manifest is a playlist that contains URLs for other .m3u8 "child" manifest files (each at a different bitrate/resolution)
  • a "child" manifest is a playlist that contains a list of video segments (usually having the .ts file extension)
• when a video is a live stream
  • the manifest contains some metadata to indicate that it's a partial playlist
  • the video player plays the video segments that are listed in the manifest
    and, when finished, re-requests the same manifest to receive an updated list of new video segments
  • when a video is "on demand"
  • the manifest contains some metadata to indicate that it's a complete playlist
  • all of the video segments for the entire video are contained in the manifest
    and the player never has a need to re-request it

IPTV terminology is confusing:

• .m3u files are playlists, and usually only contain a single .ts URL..
  which is not a short video segment but, rather, a continuous stream
  • I never intended for this type of playlist to be used with this proxy..
  • I don't see any advantage in doing so..
  • although, I suppose there could be a benefit for modifying HTTP headers,
    or possibly using hook functions to do something interesting
  • .m3u files are not parsed by the proxy as HLS manifests
  • although, appending a hash could be used to trick the proxy into doing so
    (ex: http://example.com/iptv.m3u#video.m3u8)
  • wrt major versions of the proxy..
    • v1.x and v2.x both use naive regex patterns to parse HLS manifests..
      and this trick should almost certainly work
    • v3.x uses a parser that understands the structure of HLS manifests..
      and this trick would most likely fail

with respect to testing different video players.. a simple test could be contrived using a local web server.. and monitoring a log of the requests it receives from a video player. for example:

• configure a redirect from URL 1 to URL 2
  • ex: Redirect 301 "/foo/video.m3u8" "/bar/video.m3u8"
• serve URL 2
  • ex: /bar/video.m3u8
  • configure this to be a "child" manifest for a live stream, having a few valid video segments
• after the video player plays those few video segments,
  check to see whether it re-requests URL 1 (original) or URL 2 (result of 301 redirect)

I'm not suggesting that you actually need to do any of that.. I'm only saying that it's a fairly simple test.

[warren-bank]

so.. I just reread and reconsidered everything I'd previously said after the benefit of a little sleep.. and I find myself now thinking that I want to take back everything and start again.

The short answer is: yes

The longer answer is:

• if the URL in your original question:
  https://tvnow.best/api/stream/username/password/livetv.epg/channel.name.m3u8
  is a live HLS video stream
  • then you could format it into a URL that would be requested through the proxy:
    http://localhost:8080/aHR0cHM6Ly90dm5vdy5iZXN0L2FwaS9zdHJlYW0vdXNlcm5hbWUvcGFzc3dvcmQvbGl2ZXR2LmVwZy9jaGFubmVsLm5hbWUubTN1OA==.m3u8
  • when this URL is played by any video player, all 3xx redirects are handled internally by the proxy
  • the video player is completely unaware of them
  • when the live stream is re-requested by the player from the proxy, it will re-request this same URL
  • all redirection will again be handled internally by the proxy
  • the resulting manifest URL and its content will contain fresh auth tokens

Sorry! I have no idea why I didn't simply say that before. Guess I was more tired than I thought.

Anyway, my suggestion is to give this proxy a test.. and please report back your findings.

[warren-bank]

PS: I checked some old notes and confirmed that I've tested and used "tvnow.best" in the past.. and really liked this service a lot. These streams are HLS and will work perfectly with the proxy.

[bruor]

I would be interested in comparing notes, do you have a discord server or some way we can connect outside github?

[warren-bank]

I hadn't used Discord before.. not the most "social" person ever. After taking a quick test drive, I see its value.. so I registered an account. I locked the account down pretty tightly.. but I think you should still be able to send me a direct message. My username is: "warren.bank" and the display name is "warren-bank"; not sure which you would need. If DM is disabled by my privacy settings, please let me know.. and I'll either loosen the settings, or invite you to join a private channel on my server.

[bruor]

DM is disabled, you can try me there, same username. I think you have to add me as a friend by username.