Closed Shepless closed 1 year ago
the line of code you point to is:
if ((data instanceof Object) && (data.headers instanceof Object) && (typeof data.headers['content-encoding'] === 'string')){
// ...
return decoded_data
}
return data
is the issue that the server is returning the header:
content-encoding:
so it exists.. it's a string.. but it has an empty value?
if so, would simply changing that line to also test that the value is non-empty fix your issue?
if ((data instanceof Object) && (data.headers instanceof Object) && (typeof data.headers['content-encoding'] === 'string') && data.headers['content-encoding']){
// ...
return decoded_data
}
return data
@warren-bank I shall give that a try. However, I forgot that whilst debugging I found that it wasn't hitting the modify m3u8 content function at all because the m3u8 I'm using has .ts extensions for each stream. For example:
#EXTINF:
aHR0cHM6Ly90dm5vdy5iZXN.....=.ts
This makes it skip due to the is_m3u8 check in the proxy.js. I forgot when debugging that I had forced it to modify the content when I stumbled on the content-encoding issue.
sorry.. I answered before actually reading what was inside of that if
block..
now, I see that the value of the header is tested for supported encodings.. and if the value isn't supported (or is empty).. then the decoder doesn't do anything.. so the method is harmless
can we just start at the beginning.. you can change the URL domains and paths.. but just give an idea of what you've done.. so I can quickly confirm that it isn't something simple and obvious.
what does the URL for the manifest look like.. that you are giving to a client.. for it to access a stream via the proxy?
http://<ip of proxy>:<port of proxy>/<base64 of manifest URL>.m3u8
that manifest isn't HLS..
IPTV .m3u lists of channels look a lot like HLS .m3u8 lists of video chunks..
it's confusing..
but your manifest is for IPTV, and each .ts
URL points to a stream..
whereas in HLS, each .ts
URL points to a file that holds about 10 seconds of video.
I don't really know the technical specifics.. only that the distinction is a constant source of aggrevation.
I thought that at first too, but under the hood it is an m3u8, the .ts is doing a 301 redirect to the m3u8 (annoyingly!) I added this console log in to debug it inside the request module:
const redirect = resolveRedirectLocation(error.location, original_url, redirects)
console.log('REDIRECT DETECTED FROM', original_url, 'TO', redirect);
Example Output:
REDIRECT DETECTED
FROM
https://tvnow.best/api/stream/xxx/xxxx/livetv.epg/5.star.max.eastern.us.m3u8
TO
https://195-181-172-33.servers.tips:5052/live11/Otr8WWXE98TJUfn97yfsIQ/632263/1679019690/1022584.m3u8
curl -i https://195-181-172-33.servers.tips:5052/live11/Otr8WWXE98TJUfn97yfsIQ/632263/1679019690/1022584.m3u8 HTTP/1.1 200 OK Server: nginx Date: Thu, 16 Mar 2023 22:24:36 GMT Content-Type: application/vnd.apple.mpegurl Content-Length: 252 Connection: keep-alive Last-Modified: Thu, 16 Mar 2023 22:24:25 GMT ETag: "64139719-fc" Expires: Thu, 16 Mar 2023 22:24:41 GMT Cache-Control: max-age=5 Strict-Transport-Security: max-age=31536000; includeSubdomains; Access-Control-Allow-Origin: * Access-Control-Allow-Methods: GET, POST, OPTIONS Access-Control-Allow-Headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range Access-Control-Expose-Headers: Content-Length,Content-Range
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:13
#EXT-X-MEDIA-SEQUENCE:2640
#EXTINF:11.878533,
1022584_2640.ts
#EXTINF:11.177833,
1022584_2641.ts
#EXTINF:12.212200,
1022584_2642.ts
#EXTINF:12.679333,
1022584_2643.ts
#EXTINF:12.645967,
1022584_2644.ts
i'm lost.. you said URLs with a .ts extension are redirecting to URLs with a .m3u8 extension.. but your example is .m3u8 to .m3u8
where did this come from:
https://tvnow.best/api/stream/xxx/xxxx/livetv.epg/5.star.max.eastern.us.m3u8
and.. presumably.. having cherry picked the URL for an HLS manifest for only one channel.. will it work with the proxy (if used directly)?
and.. presumably.. having cherry picked the URL for an HLS manifest for only one channel.. will it work with the proxy (if used directly)?
If I cherry pick one of this streams to proxy directly, it still fails because it isn't rewriting the m3u file contents, its just using the raw output for the chunks (/12345.ts)
ok.. I see the problem
you're feeding an EPG into the proxy in place of a manifest.
because the manifest doesn't have the tags that the parser uses to identify it as being a master
manifest,
it is seen as being a video manifest..
and all URLs in a video manifest are for video segments (ie: .ts files)
try this..
forget about the EPG..
cherry pick one channel (ex: 5 Star Max (East)
)..
and try to proxy the URL for its HLS manifest (ex: https://tvnow.best/api/stream/<user>/<password>/livetv.epg/5.star.max.eastern.us.m3u8
)
what happens?
Sorry, I had just tried that before you replied. See the above response. Apologies (and thank for all your help with this!)
If i play the non-proxy url this is what happens:
So it looks like the 301 redirect is affecting it somehow.
I think you misunderstood.. because your example that failed is still using a .ts extension.. which it shouldn't be
what I'm saying is.. use that URL (for the particular channel).. as your starting point to proxy.. and it should work
I did just that. (used the base64 encoded string for https://tvnow.best/api/stream/<user>/<password>/livetv.epg/5.star.max.eastern.us.m3u8
)
That first screen shot is when it tried to request chunks from the proxy. Here it is with more context.
the .ts chunks are 404ing because the proxy isn't modifying the m3u8 content to prepend the base64 hash. My guess is because https://tvnow.best/api/stream/<user>/<password>/livetv.epg/5.star.max.eastern.us.m3u8
does a 301 redirect to https://195-181-172-33.servers.tips:5052/live3/KJM_52qoazU4umeRv8Fhag/632263/1679022183/24005968.m3u8
?
humor me..
127.0.0.1:9002
Clappr
video player, and watch the stream in your browser
Network
tab in Chrome devtools, and the command-line logs from your proxysorry, "Clappr" is under "Chromecast sender"
choosing "HLS-Proxy configuration" is the page that I sent you to directly.. choosing it from the dropdown just loops you back in a circle
Aye sorry, got there in the end. I get CORS errors, guessing thats my proxy setup?
that should never happen with the proxy.. it always adds CORS headers
If this is any help:
that's specific to your browser and its security policy.. and that changes with every release..
you need a browser that can load an HTTP (not secure) page.. run a script from an HTTPS (secure) URL.. and XHR content from HTTP at localhost.
if you want.. you can run the proxy with https.. and load the same proxy page from here with https
otherwise, you'll need to figure out what options to change.. or flags to pass to the browser when you start it (from the command line)
ok got it working. but its getting lots of 500 errors
Im guessing you got logs on that?
don't look at me..
view-source:http://webcast-reloaded.surge.sh/4-clappr/index.html
shows that Clappr is coming from:
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/clappr@0.3.13/dist/clappr.min.js"></script>
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/level-selector@0.2.0/dist/level-selector.min.js"></script>
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/clappr-chromecast-plugin@0.1.1/dist/clappr-chromecast-plugin.min.js"></script>
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/dash-shaka-playback@3.0.3/dist/dash-shaka-playback.min.js"></script>
..that's a pretty standard CDN
But those 500s are coming from the local proxy?
oh yeah, sorry.. you're right.. the XHR request was made by Clappr (which did load).. and that's what's hitting browser security blocks
anway.. this is getting way off track
sorry: "AirPlay sender"
That works fine in VLC.
yeah, no browser security to mess things up
So the conclusion is - it works :rofl: ? Was the issue with me trying it a web based player? (Aside from the M3U beng a TVG format - which I can sort my end)
that was supposed to be the fast way to test what I was saying.. but boy, did that go off track.. anyway.. it proves that:
as I said earlier.. your screenshot that you said failed shows you trying to proxy a manifest with a .ts extension.. which made no sense. this little test proved that you can proxy an .m3u8 extension URL.. taken from within your EPG.
..but NOT the .m3u8 URL for the EPG itself!!
..because it isn't an HLS manifest
Oh man, I'm so sorry! I really appreciate your effort and patience with me on this! I've learnt a lot. Thanks again. I'll close this behemoth off.
no worries.. glad you find the proxy useful :)
Firstly, thanks for such an excellent package and all the work you have put into this @warren-bank
After spending some time (and failing!) to get my my m3u8 file to proxy correctly, I spent some time debugging it and found that if a response doesn't include a
content-encoding
header the response just returns empty, This empty response then stops themodify_m3u8_content
function from modifying the contents to prepend the proxied URL.I was hoping to fix it myself, but alas my node stream skills aren't very good so I have spent a few hours failing.
The issue isn't your library really, just poorly implemented server responses. But the issue occurs here in your request package
Update Things I tried when fixing myself:
data.toString('utf-8')
. This however just resulted in the data printing[object Object]