Proxy retrieves parent HLS file but redirects none of the children

[Artrix9095]

Hi, when working with your proxy I've come across a issue...

Lets use the playlist from #8 for an example

So lets say I were to try to get https://www.cbsnews.com/common/video/cbsn_header_prod.m3u8 from CBS news through the proxy... The URL I get is: http://127.0.0.1:8000/aHR0cHM6Ly93d3cuY2JzbmV3cy5jb20vY29tbW9uL3ZpZGVvL2Nic25faGVhZGVyX3Byb2QubTN1OA==.m3u8

Now when fetching this... I get the response back

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-STREAM-INF:BANDWIDTH=581169,RESOLUTION=640x360,FRAME-RATE=29.970,CODECS="avc1.4D401E,mp4a.40.2"
http://127.0.0.8:8000/aHR0cHM6Ly9jYnNuLXVzLmNic25zdHJlYW0uY2JzbmV3cy5jb20vb3V0L3YxLzU1YTg2NDhlOGYxMzRlODJhNDcwZjgzZDU2MmRlZWNhL21hc3Rlcl8xMS5tM3U4.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=264369,RESOLUTION=320x180,FRAME-RATE=29.970,CODECS="avc1.4D400D,mp4a.40.2"
http://127.0.0.8:8000/aHR0cHM6Ly9jYnNuLXVzLmNic25zdHJlYW0uY2JzbmV3cy5jb20vb3V0L3YxLzU1YTg2NDhlOGYxMzRlODJhNDcwZjgzZDU2MmRlZWNhL21hc3Rlcl8xMi5tM3U4.m3u8

This all works fine... but, when trying to request any of the other URLs inside the playlist I just receive connection timeouts I've tried turning on debugging mode 3, and received this in my console:

HTTP server is listening at: 127.0.0.8:8000
proxying (raw): /aHR0cHM6Ly93d3cuY2JzbmV3cy5jb20vY29tbW9uL3ZpZGVvL2Nic25faGVhZGVyX3Byb2QubTN1OA==.m3u8

proxying: https://www.cbsnews.com/common/video/cbsn_header_prod.m3u8

m3u8: true

modify (raw): {
  match: '\n' +
    'https://cbsn-us.cbsnstream.cbsnews.com/out/v1/55a8648e8f134e82a470f83d562deeca/master_11.m3u8\n',
  head: '\n',
  abs_path: 'https://',
  rel_path: 'cbsn-us.cbsnstream.cbsnews.com/out/v1/55a8648e8f134e82a470f83d562deeca/',
  file_name: 'master_11',
  file_ext: '.m3u8',
  tail: '\n'
}

redirecting: https://cbsn-us.cbsnstream.cbsnews.com/out/v1/55a8648e8f134e82a470f83d562deeca/master_11.m3u8

redirecting (proxied): http://127.0.0.8:8000/aHR0cHM6Ly9jYnNuLXVzLmNic25zdHJlYW0uY2JzbmV3cy5jb20vb3V0L3YxLzU1YTg2NDhlOGYxMzRlODJhNDcwZjgzZDU2MmRlZWNhL21hc3Rlcl8xMS5tM3U4.m3u8

modify (raw): {
  match: '\n' +
    'https://cbsn-us.cbsnstream.cbsnews.com/out/v1/55a8648e8f134e82a470f83d562deeca/master_12.m3u8',
  head: '\n',
  abs_path: 'https://',
  rel_path: 'cbsn-us.cbsnstream.cbsnews.com/out/v1/55a8648e8f134e82a470f83d562deeca/',
  file_name: 'master_12',
  file_ext: '.m3u8',
  tail: ''
}

redirecting: https://cbsn-us.cbsnstream.cbsnews.com/out/v1/55a8648e8f134e82a470f83d562deeca/master_12.m3u8

redirecting (proxied): http://127.0.0.8:8000/aHR0cHM6Ly9jYnNuLXVzLmNic25zdHJlYW0uY2JzbmV3cy5jb20vb3V0L3YxLzU1YTg2NDhlOGYxMzRlODJhNDcwZjgzZDU2MmRlZWNhL21hc3Rlcl8xMi5tM3U4.m3u8

Do you have any idea why this is happening? Does it have something to do with the fact that I'm using hls.js? Or is this an internal error? Or maybe its just me. Any help is appreciated, thanks.

[warren-bank]

quick thoughts:

• I just tested the demo hls video by:
  • configuring a URL to an instance of the hls proxy using this page without any referer
  • playing the resulting URL with Clappr video player on this page
  • which can be accessed directly using this permalink
  • and it works correctly
• it seems clear that the problem is with the video player..
  • your description of your error includes:
  • a log from the proxy server, which only includes the requests that it receives.. and everything is OK
  • a screenshot from your browser, which indicates that some requests are not reaching the proxy server..
    • could you copy the URL of a failing request and paste it here?
    • so we can inspect its format.. and try to figure out why it isn't being sent to the proxy server
    • if it's an m3u8, then it should be exactly equal to either:
      • 640x360
      • 320x180
    • either of which, you can request directly.. in either your web browser or curl.. to confirm that the server will respond with a child manifest for the specific video resolution
    • likewise, you can continue to drill down into each.. and manually request individual video segments.. to confirm that:
      1. they are directed to the proxy server
      2. the proxy server responds correctly
    • to do so.. takes all of the client code out of the equation.. and you can test that the backend is working as it should
      • then.. it becomes a matter of debugging the video player client (which is outside of the scope for this issue)

[warren-bank]

PS: if the video player webpage (that uses hls.js) is hosted on https, then the problem is most likely that the proxy server is currently only running http.. and browser security policy won't allow requesting insecure resources. If that is the case, then you could either load your webpage with http, or run your proxy server with https.. or simply use a web browser that doesn't enforce this policy.

[warren-bank]

PPS: I'm a little confused by one thing.. could you please clarify..

Are you running your instance of the proxy server at:

• 127.0.0.1:8000 or
• 127.0.0.8:8000

I see both hosts mentioned in your description.. so I'm wondering if that's the actual cause for your problem?

[Artrix9095]

Hi thank you for the zippy responses, Its 127.0.0.8

[Artrix9095]

here is one of the urls that are trying to be requested... http://127.0.0.8:8000/aHR0cHM6Ly9jYnNuLXVzLmNic25zdHJlYW0uY2JzbmV3cy5jb20vb3V0L3YxLzU1YTg2NDhlOGYxMzRlODJhNDcwZjgzZDU2MmRlZWNhL21hc3Rlcl8xMS5tM3U4.m3u8

[Artrix9095]

I just noticed... there seems to be some inconsistency between 127.0.0.1 and 127.0.0.8... in the initial request it goes to 127.0.0.1, but in all the others it goes to 127.0.0.8. When I ran the proxy, I specified that the hostname should be 127.0.0.8 which makes this very confusing.

[warren-bank]

dumb question.. but does 127.0.0.8 also resolve to the machine running the hls proxy?

[Artrix9095]

Incase your wondering, here is a snippet of my react component

const videoProps = {
        controls: true,
        width: '100%',
        height: '100%',
        playerRef: videoRef,
        src: `${proxyUrl}/${btoa('https://www.cbsnews.com/common/video/cbsn_header_prod.m3u8')}.m3u8`,
        hlsConfig: {
            maxLoadingDelay: 4,
            minAutoBitrate: 0,
            lowLatencyMode: true,
            xhrSetup: console.log,
        },
    };
    return (
        <>
            <Player {...videoProps} />
        </>
    );

[Artrix9095]

I don't think it does, if i switch the proxy url to 128.0.0.8 the request just fail, while if its 128.0.0.1 the initial requests work fine

[Artrix9095]

But I've noticed that the child requests are all using 127.0.0.8 instead of 127.0.0.1

[Artrix9095]

I'm dumb if you see 128 I mean 127

[Artrix9095]

I think I just got it to work, you have to use localhost when using the proxy locally, npm start --host "localhost"

[warren-bank]

well.. the only purpose for the --host cli option is to redirect the internal URL back through the proxy.

since you're currently running the proxy server on localhost.. you can just use: --host 127.0.0.1 ..which is the same as: --host localhost

yep.. you beat me to the "comment" button :) ..you got it

[Artrix9095]

Thank you for the fast responses man, you helped me realize the problem, when running with --host 127.0.0.1, for some reason it doesn't work when I host it locally...

[warren-bank]

no worries..

out of curiousity.. where did the IP 127.0.0.8 come from?

was it just from a typo on the command line?: --host 127.0.0.8 ..or did the proxy server (incorrectly) determine that this was the IP for one of your network interfaces all on its own?