warren-bank / HLS-Proxy

Node.js server to proxy HLS video streams
http://webcast-reloaded.surge.sh/proxy.html
GNU General Public License v2.0
238 stars 68 forks source link

feature request: more secured video url source #39

Closed siberkolosis closed 7 months ago

siberkolosis commented 7 months ago

hope HLS-Proxy have this feature. i have implemented to website. people can easy to decode base64 to view url resource.

thank you.

warren-bank commented 7 months ago

hmm.. this could be done if I was to add a new hook function: rewrite

for example:

  hlsd -v 3 --port 8080 --hooks "/path/to/hooks.js"

where:

  // hooks.js

  const Cryptr = require('cryptr')
  const cryptr = new Cryptr('my secret symmetric AES encryption key')

  const encrypted_url_prefix = 'http://encrypted.example.com/'

  const encrypt_url = (cleartext_url) => {
    const encrypted_url = (cleartext_url)
      ? encrypted_url_prefix + cryptr.encrypt(cleartext_url)
      : ''

    return encrypted_url
  }

  const decrypt_url = (encrypted_url) => {
    let cleartext_url = encrypted_url

    if (encrypted_url && encrypted_url.startsWith(encrypted_url_prefix)) {
      encrypted_url = encrypted_url.substring(encrypted_url_prefix.length, encrypted_url.length)

      cleartext_url = cryptr.decrypt(encrypted_url)
    }

    return cleartext_url
  }

  module.exports = {
    "redirect": (cleartext_url) => encrypt_url(cleartext_url),
    "rewrite":  (encrypted_url) => decrypt_url(encrypted_url)
  }
siberkolosis commented 7 months ago

sorry i mean . proxy_url='https://mysite.com'; \ video_url="https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8"; \ file_extension='.m3u8'; \ hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension"; \ echo $hls_proxy_url;

will generate. https://mysite.com/aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4.m3u8

aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4 can easily decoded back to https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8

i dont want people can see video url source. including the URL inside master m3u8 playlist.

warren-bank commented 7 months ago

v3.4.3 adds support for this hook function, as previously described.

this recipe is verbatim from my last comment. It works perfectly.. to obfuscate all URLs.

To be clear, if a visitor to your website was to base64 decode the data being passed between video player client and hls-proxy.. they'd only see encrypted URLs. Without the symmetric key used by your hook functions (redirect to encrypt, and rewrite to decrypt).. it's completely meaningless.

siberkolosis commented 7 months ago

yes. it works perfect. another one that I really need.

proxy_url='https://mysite.com/'; video_url="https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8"; file_extension='.m3u8'; hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension"; echo $hls_proxy_url;

how to encrypt result echo $hls_proxy_url because this generated url is clear base64 which I pasted in my site's web player input.

is good if hlsd have built in paramater or command to generate ecrypted URL to HLS Proxy .

warren-bank commented 7 months ago

the simplest way would be to use the same hook function from the command-line:

proxy_url='https://mysite.com/'
video_url="https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8"
file_extension='.m3u8'
hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension"

# hooks_js='https://mysite.com//aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4.m3u8'

hooks_js='/path/to/hooks'
hls_proxy_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.rewrite('${hls_proxy_url}'));")
echo "$hls_proxy_url"
siberkolosis commented 7 months ago

cool.. thank you very much....

siberkolosis commented 7 months ago

got this error..

node:internal/crypto/cipher:122 this[kHandle].initiv(cipher, credential, iv, authTagLength); ^

TypeError: Invalid initialization vector at Decipheriv.createCipherBase (node:internal/crypto/cipher:122:19) at Decipheriv.createCipherWithIV (node:internal/crypto/cipher:141:3) at new Decipheriv (node:internal/crypto/cipher:295:3) at Object.createDecipheriv (node:crypto:149:10) at Cryptr.decrypt (/home/jmpr/node_modules/cryptr/index.js:72:33) at decrypt_url (/home/jmpr/hls/hook.js:20:30) at Object.rewrite (/home/jmpr/hls/hook.js:28:36) at [eval]:1:77 at Script.runInThisContext (node:vm:123:12) at Object.runInThisContext (node:vm:299:38) { code: 'ERR_CRYPTO_INVALID_IV' }

Node.js v18.17.1

it works if I use a different url to the one in hooks.js . but echo "$hls_proxy_url" still base64 still showing aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4

proxy_url='https://mysite.com/' video_url="https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8" file_extension='.m3u8' hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension"

hooks_js='/home/jmpr/hls/hook.js' hls_proxy_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.rewrite('${hls_proxy_url}'));") echo "$hls_proxy_url"

https://mysite.com//aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4.m3u8

warren-bank commented 7 months ago

yeah.. that's probaby because my code was incorrect.. the right idea.. but rushed.. and I messed up. sorry.

here's how it should go:

hooks_js='/path/to/hooks'
cleartext_video_url='https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8'
encrypted_video_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.redirect('${cleartext_video_url}'));")

echo "$encrypted_video_url"
# http://encrypted.example.com/f7b1c95b2eb8ff8681833ecb443ac73039e0f31937971f5874c58adff80d1f310b77ce3ac2e3a31f02bfa9ac11a16566e93f07b7f4f5aace133a28d1dfad9d93f570354c45b857478608bd706ced98438565116c53bddfea725d501361ab8067e072d320fe9e32d05d78b3c890cef9243df6e292b9a192c017089ff2d7d6ce7da7f075fa59f3764b161a4c4d8cc03cdeffd06b915da78546cb1412a3b75c6e

proxy_url='https://mysite.com'
video_url="$encrypted_video_url"
file_extension='.m3u8'
hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension"

echo "$hls_proxy_url"
# https://mysite.com/aHR0cDovL2VuY3J5cHRlZC5leGFtcGxlLmNvbS9mN2IxYzk1YjJlYjhmZjg2ODE4MzNlY2I0NDNhYzczMDM5ZTBmMzE5Mzc5NzFmNTg3NGM1OGFkZmY4MGQxZjMxMGI3N2NlM2FjMmUzYTMxZjAyYmZhOWFjMTFhMTY1NjZlOTNmMDdiN2Y0ZjVhYWNlMTMzYTI4ZDFkZmFkOWQ5M2Y1NzAzNTRjNDViODU3NDc4NjA4YmQ3MDZjZWQ5ODQzODU2NTExNmM1M2JkZGZlYTcyNWQ1MDEzNjFhYjgwNjdlMDcyZDMyMGZlOWUzMmQwNWQ3OGIzYzg5MGNlZjkyNDNkZjZlMjkyYjlhMTkyYzAxNzA4OWZmMmQ3ZDZjZTdkYTdmMDc1ZmE1OWYzNzY0YjE2MWE0YzRkOGNjMDNjZGVmZmQwNmI5MTVkYTc4NTQ2Y2IxNDEyYTNiNzVjNmU=.m3u8

tested with: node v16.14.0

warren-bank commented 7 months ago

and.. just to be clear.. what happens when this is requested by a client.. is that the proxy will do:

  const encoded_url = 'aHR0cDovL2VuY3J5cHRlZC5leGFtcGxlLmNvbS9mN2IxYzk1YjJlYjhmZjg2ODE4MzNlY2I0NDNhYzczMDM5ZTBmMzE5Mzc5NzFmNTg3NGM1OGFkZmY4MGQxZjMxMGI3N2NlM2FjMmUzYTMxZjAyYmZhOWFjMTFhMTY1NjZlOTNmMDdiN2Y0ZjVhYWNlMTMzYTI4ZDFkZmFkOWQ5M2Y1NzAzNTRjNDViODU3NDc4NjA4YmQ3MDZjZWQ5ODQzODU2NTExNmM1M2JkZGZlYTcyNWQ1MDEzNjFhYjgwNjdlMDcyZDMyMGZlOWUzMmQwNWQ3OGIzYzg5MGNlZjkyNDNkZjZlMjkyYjlhMTkyYzAxNzA4OWZmMmQ3ZDZjZTdkYTdmMDc1ZmE1OWYzNzY0YjE2MWE0YzRkOGNjMDNjZGVmZmQwNmI5MTVkYTc4NTQ2Y2IxNDEyYTNiNzVjNmU='
  const decoded_url = atob(encoded_url)

  console.log(decoded_url)
  // http://encrypted.example.com/f7b1c95b2eb8ff8681833ecb443ac73039e0f31937971f5874c58adff80d1f310b77ce3ac2e3a31f02bfa9ac11a16566e93f07b7f4f5aace133a28d1dfad9d93f570354c45b857478608bd706ced98438565116c53bddfea725d501361ab8067e072d320fe9e32d05d78b3c890cef9243df6e292b9a192c017089ff2d7d6ce7da7f075fa59f3764b161a4c4d8cc03cdeffd06b915da78546cb1412a3b75c6e

  const cleartext_url = hooks.rewrite(decoded_url)

  console.log(cleartext_url)
  // https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8

  // request from origin server, modify URLs in manifest, serve..
  // conditionally prefetch some media segments..
siberkolosis commented 7 months ago

very perfect. thank you for taking the time....

siberkolosis commented 7 months ago

not work if source url have path like this. without hooks.js , its work.

hls/mykr/index.m3u8?hdnts=st=1700725590~exp=1700772383~acl=/hls/ipfmix/~id=f655d1f39e42355f15fcaef898bf7af3~data=UmFXQzJ1RExhaGtXb29zajE4UTVCSHlzeEhTdUhnN292V0tKdHFLWmM5UEpFQUp1MXB2OUFjJTJCazVYTEdkakdK~hmac=bbd303937d081f9a7721929d56ed3e9baaafc4dcc777e06711d91fc29c62b777&hdnts2=st=1700725590~exp=1700772383~acl=/hls/ipfmix/~id=f655d1f39e42355f15fcaef898bf7af3~data=UmFXQzJ1RExhaGtXb29zajE4UTVCSHlzeEhTdUhnN292V0tKdHFLWmM5UEpFQUp1MXB2OUFjJTJCazVYTEdkakdK~hmac=bbd303937d081f9a7721929d56ed3e9baaafc4dcc777e06711d91fc29c62b777

warren-bank commented 7 months ago

I assume you truncated the URL and just didn't include the protocol and host and some of its path in this issue, but you did include them when using the proxy.

Here is a test:

#!/usr/bin/env bash

original_url='https://cph-msl.akamaized.net/foo/bar/baz/hls/mykr/index.m3u8?hdnts=st=1700725590~exp=1700772383~acl=/hls/ipfmix/*~id=f655d1f39e42355f15fcaef898bf7af3~data=UmFXQzJ1RExhaGtXb29zajE4UTVCSHlzeEhTdUhnN292V0tKdHFLWmM5UEpFQUp1MXB2OUFjJTJCazVYTEdkakdK~hmac=bbd303937d081f9a7721929d56ed3e9baaafc4dcc777e06711d91fc29c62b777&hdnts2=st=1700725590~exp=1700772383~acl=/hls/ipfmix/*~id=f655d1f39e42355f15fcaef898bf7af3~data=UmFXQzJ1RExhaGtXb29zajE4UTVCSHlzeEhTdUhnN292V0tKdHFLWmM5UEpFQUp1MXB2OUFjJTJCazVYTEdkakdK~hmac=bbd303937d081f9a7721929d56ed3e9baaafc4dcc777e06711d91fc29c62b777'

hooks_js='/path/to/hooks'
encrypted_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.redirect('${original_url}'));")
cleartext_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.rewrite('${encrypted_url}'));")

if [ ! "$original_url" == "$encrypted_url" ];then
  echo 'encryption: OK'
else
  echo 'encryption: FAIL'
fi

if [ "$original_url" == "$cleartext_url" ];then
  echo 'roundtrip: OK'
else
  echo 'roundtrip: FAIL'
fi

What exactly is the issue? My test results are:

encryption: OK
roundtrip: OK
warren-bank commented 7 months ago

I added a test bash script to the "recipe" for what we've been discussing.. so it's easy to run, and easy to test (just edit the bash script to configure your own video urls, etc).

warren-bank commented 7 months ago

Doing a little Sunday morning housekeeping.. and marked this issue as closed.

If you can find a video URL to plug into the test script that results in a failing result, then please let me know.. and I'll take another look.

siberkolosis commented 7 months ago

i try from another source video url. its work. not work if using hooks.js for any video or vod for motogp.com you can try with any video on motogp.com just change mpd extension to m3u8.

warren-bank commented 7 months ago

you were correct that:

you were incorrect that:

I believe your issue is that:

warren-bank commented 7 months ago

observations:

siberkolosis commented 7 months ago

yes it play properly when not using hooks.js to encrypt url. video dont play if using hooks.js to ecrypts url. it happen only on motogp.com

i setup hls proxy behind nginx with this command on server hlsd --hooks "hooks.js" --tls --port 8081 --manifest-extension "m3u8" --segment-extension "pict

generate inside playlist look like this.

EXTM3U

EXT-X-VERSION:3

EXT-X-TARGETDURATION:6

EXT-X-MEDIA-SEQUENCE:3806738

EXTINF:6.000000,

https://mysite.com/aHR0cHM6Ly9zdHYuYmlvL2JkYzMxZmZmMWQwYjgzMTMxY2VjY2RmYjg0YWIxM2M0OTlmODY0M2ZhNDY2MDg3OTkwZWNkNmE2ODY5YzUzMTgwZWM0Y2EzZDNhYmNkMDQwZTM4YjJlN2Q5NjY5MmNiZThkNzQ3YTU3MDE1YTE3ZTZhYTdiZTU5YWMxZDdmOTA5NTAzMDBiOTgwMmMwNTQ5ZGZhZDIwZDM1YjI1NjY3YWUzNTNlMWY0OTlkMjBlZTFkOTI4MjdhZmJiYmNiMjM4NjcwNWE4M2NiMWNhYmY4ZjY1MDdjNGQ2ODMyYjdjMzA5OTdiMzVjNDk2ODhlMjExODYwMzRjY2NjZWRmNWI0MTk5ODAxNWZiMTcwYmYwODhlZmJhZmMyMGVkMWRjZGRhNmMyZTk1ZmE2YjFhZDUzM2U4NWYzZmQxNDk4YTc3OWU5YTlhY2E0MzUwNzAzOGMxM2MwZGZmODA3MWQ3MjUxM2M1M2Y5NTc1YQ==.pict

EXTINF:6.000000,

https://mysite.com/aHR0cHM6Ly9zdHYuYmlvLzQxOGIwOWMxOWViZjAwMjdjZGJhNmIzM2U3ZGVlZGQxN2Q3MDM0Zjk1NmQ0ZWZhNjdhNTQ3M2Y3NTE4NGRlNmYwNjZmNTM0ZmIwYmFkN2NjOTI4ZTA1MWJhYmE0MTc0ZDUwNjE0NTYxNTgzNzUxNTY4ZTFlZjE4N2Y1NjA1ZjQxNWFkZWNjMjEwZWYzOWY4NWI1ZTllM2FjNzFiZjk4MmJiMmQ3Mzg2NzUxZGQxMmZmZGNlNGJmYjk0NGNkOTg5MTg1YjUyN2Q2M2JjMzk1N2QzNmNiZWY2N2JkNmQ3Y2QxNjg0ZTJiM2YyNmQyNDMzYzBmMWVkZWExMDQ3NzM3YzUzYzNmNTc2NGZlMzc4OGQ3OGJiMTVlMjQwMGE0OWVhZDU1NDljZjk3Y2UyMGJmNTk3N2M3ODJkYjc3ODNjOGVhYzdmMmY1MzE2ZmRiODkwODNjNDk3ZmFlNmNkOGY4NDkxNjA3ZDlkZA==.pict

EXTINF:6.000000,

https://mysite.com/aHR0cHM6Ly9zdHYuYmlvL2U5NzFkYjU0ZmI0NTk0MWExYzEwOTRlNWM0ZTc0OGMxMWVjMjRjOGJiZTViYTU2ODM1ZTI2ZjNjNzllMmNmMTZhOWI4MWFjMGZjNzE3MGZjYmVjNTA3MmVkNzlkNWY5YzliYmExNzYyMWY2MzNkNDA1ZWExYzhiZGMxNzE3Yzc4NjYxODJiMmVhNjU4MjA2NGQzZjU1NTk4OWFmMDhiZWFmM2MyNTNkMGViY2QxYzZlZDViOTliMDhkYzQ0ZGJmNzA3ZDYxNTk5NWZmYmFmNTc2M2QxZjRmNmY0MWI5NTZhODVhZjhkMjg4Y2Y2OGY2MmJjNzE1ZGMwMTVkYjhiZWRmNWVhZTQwOWNlYzM4YzE1NjM1ODZlNDVhMTczNWFiOTUzNGIwNWZjY2ExNzlmMzljNzZlYjkxNzdmNWYyNzdhODIxODEwMTUwNmZmZDg3MTE5YTQwYzUxNWYwOTRjZjAzZjc2NzZlMw==.pict

there's something else I want i want to set url domain inside playlist
changed from mysite.com to https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=2592000&container=focus&gadget=a&url=https://mysite.com so will be look like this

EXTM3U

EXT-X-VERSION:3

EXT-X-TARGETDURATION:6

EXT-X-MEDIA-SEQUENCE:3806738

EXTINF:6.000000,

https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=2592000&container=focus&gadget=a&url=https://mysite.com /aHR0cHM6Ly9zdHYuYmlvL2U5NzFkYjU0ZmI0NTk0MWExYzEwOTRlNWM0ZTc0OGMxMWVjMjRjOGJiZTViYTU2ODM1ZTI2ZjNjNzllMmNmMTZhOWI4MWFjMGZjNzE3MGZjYmVjNTA3MmVkNzlkNWY5YzliYmExNzYyMWY2MzNkNDA1ZWExYzhiZGMxNzE3Yzc4NjYxODJiMmVhNjU4MjA2NGQzZjU1NTk4OWFmMDhiZWFmM2MyNTNkMGViY2QxYzZlZDViOTliMDhkYzQ0ZGJmNzA3ZDYxNTk5NWZmYmFmNTc2M2QxZjRmNmY0MWI5NTZhODVhZjhkMjg4Y2Y2OGY2MmJjNzE1ZGMwMTVkYjhiZWRmNWVhZTQwOWNlYzM4YzE1NjM1ODZlNDVhMTczNWFiOTUzNGIwNWZjY2ExNzlmMzljNzZlYjkxNzdmNWYyNzdhODIxODEwMTUwNmZmZDg3MTE5YTQwYzUxNWYwOTRjZjAzZjc2NzZlMw==.pict

how to do this? already try adding --host option in hlsd parameter. dont work. just get a messed up url inside playlist.

warren-bank commented 7 months ago

a few quick notes:

related update:

siberkolosis commented 7 months ago

when you ran this, was "hooks.js" in your current working directory? yes. i try use https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8, it working. not work when use m3u8 vod from motogp.com, for motogp live event streaming, I haven't tried it. MotoGP live event has ended now. are you already test use motogp.com with hooks.js url encrypts?

thanks for new hooks function: redirect_final(url). works perfect. but not work when i add parameter --prefetch --cache-timeout 3600 --max-segments 10

warren-bank commented 7 months ago

I just played around for a few minutes with videos hosted on motogp.com, found that:

siberkolosis commented 7 months ago

module.exports = { "redirect_final": (url) => https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=4&container=focus&gadget=a&url=${encodeURIComponent(url)} }

not work. if i add parameter --prefetch --cache-timeout 3600 --max-segments 10

using google proxy. i can get good connection in my country. and prevent blocking my domain on their dns.

warren-bank commented 7 months ago

this is interesting..

test.sh:

#!/usr/bin/env bash

url='https://cdn1vodmcrdornacom.akamaized.net/112759/index.m3u8?hdnts=st=1701327912~exp=1701328374~acl=/112759/...'
ref='https://www.motogp.com/en/videos/2023/11/29/mir-talks-crew-chief-switch-im-really-happy-with-the-decision/485682?playlistId=425969'
agt='Chrome/100'

curl -s -I "$url"

curl -s -I --referer "$ref" "$url"

curl -s -I --user-agent "$agt" --referer "$ref" "$url"

and logging its output:

./test.sh >test.log 2>&1

the log file doesn't confirm my earlier observation:

HTTP/1.1 200 OK
Server: nginx/1.22.1
X-Amz-Cf-Pop: LAX53-P4
X-Amz-Cf-Id: RLKG4rWDo4p04-2WLCkoHQWpBDQgEiAaIcBUCU3kgAIrjIK6-0cjyA==
Expires: Thu, 30 Nov 2023 07:07:18 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 30 Nov 2023 07:07:18 GMT
Connection: keep-alive
Akamai-Mon-Iucid-Del: 1291036
Alt-Svc: h3-Q050=":443"; ma=93600,quic=":443"; ma=93600; v="46,43"
Content-Type: application/x-mpegURL
Content-Length: 8547

HTTP/1.1 200 OK
Server: nginx/1.22.1
X-Amz-Cf-Pop: LAX53-P4
X-Amz-Cf-Id: uskDWS1BqGQ-2qPFaSfj57usSzR5WzWlmvGVkwGEecfMKrxZW1RMbw==
Expires: Thu, 30 Nov 2023 07:07:18 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 30 Nov 2023 07:07:18 GMT
Connection: keep-alive
Akamai-Mon-Iucid-Del: 1291036
Alt-Svc: h3-Q050=":443"; ma=93600,quic=":443"; ma=93600; v="46,43"
Content-Type: application/x-mpegURL
Content-Length: 8547

HTTP/1.1 200 OK
Server: nginx/1.22.1
X-Amz-Cf-Pop: LAX53-P4
X-Amz-Cf-Id: _QvY-2CCw1lR814kAGqrsOJ2ZUmFfaoib4IvRzWZvfOg_MoKSMCD5A==
Expires: Thu, 30 Nov 2023 07:07:19 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 30 Nov 2023 07:07:19 GMT
Connection: keep-alive
Akamai-Mon-Iucid-Del: 1291036
Alt-Svc: h3-Q050=":443"; ma=93600,quic=":443"; ma=93600; v="46,43"
Content-Type: application/x-mpegURL
Content-Length: 8547

which makes me think that when HLS Proxy is using the rewrite hook to decrypt.. the referer URL isn't getting extracted from the encoded URL properly.. TBD

warren-bank commented 7 months ago

I'm not familiar with that google proxy.. so I just quickly tested it.. with the URL for CBS News master manifest.. and the response was:

<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"/><title>Sorry...</title><style> body { font-family: verdana, arial, sans-serif; background-color: #fff; color: #000; }</style></head><body><div><table><tr><td><b><font face=sans-serif size=10><font color=#4285f4>G</font><font color=#ea4335>o</font><font color=#fbbc05>o</font><font color=#4285f4>g</font><font color=#34a853>l</font><font color=#ea4335>e</font></font></b></td><td style="text-align: left; vertical-align: bottom; padding-bottom: 15px; width: 50%"><div style="border-bottom: 1px solid #dfdfdf;">Sorry...</div></td></tr></table></div><div style="margin-left: 4em;"><h1>We're sorry...</h1><p>... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.</p></div><div style="margin-left: 4em;">See <a href="https://support.google.com/websearch/answer/86640">Google Help</a> for more information.<br/><br/></div><div style="text-align: center; border-top: 1px solid #dfdfdf;"><a href="https://www.google.com">Google Home</a></div></body></html>
siberkolosis commented 7 months ago

its working. u can test with this player. https://players.akamai.com/players using google proxy only work on web player. may you need custom header to play on external player. i don't know.

https://players.akamai.com/players/hlsjs?streamUrl=https%3A%2F%2Fimages-blogger-opensocial.googleusercontent.com%2Fgadgets%2Fproxy%3Frefresh%3D4%26container%3Dfocus%26gadget%3Da%26url%3Dhttps%3A%2F%2Fwww.cbsnews.com%2Fcommon%2Fvideo%2Fcbsn_header_prod.m3u8

warren-bank commented 7 months ago

..but that would explain why it breaks prefetch ..if HLS Proxy isn't allowed to request these URLs directly ..you may need to configure some additional request headers (on the command line)

siberkolosis commented 7 months ago

tested it using curl. its working just use random referrer -H 'Referer: https://www.nodfgdf.vom' or anything

warren-bank commented 7 months ago

ok.. the Google proxy does require a Referer request header.. it looks like any value will do

ex:

#!/usr/bin/env bash

url='https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=4&container=focus&gadget=a&url=https%3A%2F%2Fwww.cbsnews.com%2Fcommon%2Fvideo%2Fcbsn_header_prod.m3u8'
ref='https://www.google.com/'

curl -s -I "$url"
# 403

curl -s -I --referer "$ref" "$url"
# 200
warren-bank commented 7 months ago

same time :smiley:

so.. you can set a default Referer header in HLS Proxy with the option:

--referer 'http://example.com/'

..and that should allow prefetch to work with this Google proxy

or.. after I've figured out what's going on with encryption/decryption.. combined with passing a referer URL along with video URL in the base64 encoded value.. and fix it.. then that should work too

siberkolosis commented 7 months ago

yes, its working now. thank you.

i am using cloudflare cdn with custom ts extension name. , and then using google proxy. its works perfect.

just need optimize nginx conf. already tested 8k viewer live streaming. my server only use 1gbps. the problem is cloudflare need first request to get cached. but, with many viewer first request ts from m3u8 in same time. many get cache-status MISS. miss status that mean fresh content before cf cached content. my cf rule only cached for ts file. my trick is i restarted nginx every 10 or 5 second using loop in terminal. to stop ,many cloudflare connection because hit my 1gbps my server connection . i think if few person is first request. and then all viewer can delayed. it should all viewer will hit cached content on cloudflare. or may 1gbps server it's not enough.

warren-bank commented 7 months ago

v3.4.5 fixes the issue having to do with combining (video + referer urls) with (encryption/decryption).

the order of events needed correction.

the way it works now (correctly) is:

warren-bank commented 7 months ago

note: that this last update effects how to construct the initial URL

siberkolosis commented 7 months ago

encryption/decryption avoid stranger people use my proxy..

sorry, can you help write hooks.js, combining with google proxy.

warren-bank commented 7 months ago
  1. copy this
  2. edit hooks.js
    • add to module.exports:
      "redirect_final": (url) => `https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=4&container=focus&gadget=a&url=${encodeURIComponent(url)}`
siberkolosis commented 7 months ago

already updated to v3.4.5, i still no success to play video hosted on motogp.com with hooks.js encryption.