warren-bank / Android-WebCast

Android app to extract video (file/stream) URLs from websites and watch them elsewhere (internal/external video player, Google Chromecast, ExoAirPlayer).
GNU General Public License v2.0
97 stars 17 forks source link

add headers #6

Closed kevin4dhd closed 3 years ago

kevin4dhd commented 3 years ago

how to add headers? any example? please

warren-bank commented 3 years ago

There is no support for adding custom request headers.

A 'Referer' header (that refers to the URL of the webpage on which the video was identified) is sent both: (a) to the internal video player, (b) in the Intent used to start an external video player as a String extra in the field "referUrl".

The reason that all of the original request headers aren't included in the Intent is due to the use of WebView listeners to identify requested video files; there is no API to notify a listener of events containing HTTP request headers.. so there's no way to intercept and copy them.

kevin4dhd commented 3 years ago

Sorry if I open this thread again, but I don't know what to do to make it work anymore, the application https://play.google.com/store/apps/details?id=com.instantbits.cast.webvideo has the option to send you headers, I only need to send a header which is this "User-Agent", "Lavf / 57.83.100" for my client's server to work but I have tried everything, I have tried with cast receiver and no results, I am thinking of using a proxy but I would like to know if this app also uses proxy, or uses the cast v2 to make it work, because if I send a header to this app it plays the video and sends it to the cast correctly, I am more interested in the cast and I get it to play in it exoplayer, but as much as I have done everything I have not managed to send that header to the cast, any suggestion would be of great help.

String videoURL = url; Intent shareVideo = new Intent(Intent.ACTION_VIEW); shareVideo.setDataAndType(Uri.parse(videoURL), "video/*"); shareVideo.setPackage("com.instantbits.cast.webvideo"); Bundle headers = new Bundle(); headers.putString("Cookie", "some cookie"); headers.putString("User-Agent", "Lavf/57.83.100"); shareVideo.putExtra("android.media.intent.extra.HTTP_HEADERS", headers); shareVideo.putExtra("headers", headers); shareVideo.putExtra("secure_uri", true) startActivity(shareVideo);

warren-bank commented 3 years ago

thoughts:

  1. I would start by confirming your assertion that the video host requires that one distinct User-Agent string to serve content
    • that's really odd
    • if true, then the video should not play in the WebCast WebView.. because it hard-codes the User-Agent string to Chrome 70 for Windows 10 desktop
    • I would run a few tests on the command-line using curl to modify the request headers.. to see what works and what doesn't
  2. If you're absolutely certain, then you can:
    • use WebCast to identify the URL for the video stream (though it won't play) on a web page
    • share this video URL by starting an implicit Intent
    • starting the Intent in a video player that allows you to configure the User-Agent request header for network streams
      • off-hand, I don't know of one that provides this feature.. but that's only because it's not a feature that I use.. it seems like a feature that is easy enough to implement that many/most probaby would offer the option
  3. PS: if you're right, then there's no way to watch the video stream on Chromecast without using a proxy server that will add the necessary request header(s)
    • HLS Proxy would be useful for this:
      hlsd --port "8080" --req-insecure --useragent "Lavf/57.83.100" -v 1
kevin4dhd commented 3 years ago

Maybe what I say doesn't make sense, I'm not an expert on this, but your HLS-Proxy library works on servers, doesn't it? It means that I have to buy a vps to be able to install the library and that there I can install the headers, tell me if I am correct, I cannot imagine the cost of bandwidth since my client has thousands of users so the server would stop to work, tell me if your library can be used in Android so Android itself is the proxy that runs in the cast, as well as NanoHttpd, I was investigating all night but it seems that it is very difficult to use that library and only when you want to add some headers to your url, please tell me if your library can be used in android as a local I would be totally grateful

kevin4dhd commented 3 years ago

Web Video Caster when transmitting to my computer the video that the header needs, what it does is use my ip (192.168.1.7), and it uses it as a proxy, I think this would be the only way it works

image

WhatsApp Image 2021-05-09 at 6 56 45 AM

kevin4dhd commented 3 years ago

This is the url that is executed to make it work, I am totally sure that the same url sends it to the cast to make it work, do you have any idea to achieve that? If I manage to do it, I will share the code with you, so maybe later make a library for new users who want to send headers in their url to cast http://192.168.1.7:30001/proxy/LM8YibV5/movie/5Ip37HpXoc/j1xKakGiX4/1127.mp4 image

warren-bank commented 3 years ago

your screenshot that shows Chrome devtools indicates that your video is not an HLS stream but, rather, a static mp4 video file; this greatly simplifies your task. Though HLS Proxy is intended for use with HLS streams.. because it has logic to modify manifest files inflight.. it could still be used to proxy static files and add/modify request headers.. and it's only a javascript app written to run in node.js.. which could be run locally on the phone.. but since your use-case is so simple.. there should be a wealth of other tools that could do what you need. All you need is a video player that support configuring its User-Agent string. Heck, there are web browsers for Android that support playing mp4 video and provide the option to configure User-Agent.. so strictly speaking.. you should be able to play your video file directly in such a web browser.

But (to be clear).. even if you find a video player app that will satisfy your video hosts request header requirement.. which will allow the video to play on Android.. this will not allow you to watch the video on Chromecast because the video player that runs in the receiver app on the Chromecast uses its own default User-Agent.. you would need to research if/how to change this header from within a custom Chromecast receiver app.. and then cast the video from an Android app that allows you to configure the ID of the Chromecast receiver app that it casts to.. which would allow you to enter the ID of your customized app. PS: WebCast hard-codes the ID to refer to its own (vanilla) Chromecast receiver app.

warren-bank commented 3 years ago

lets rewind this conversation.. is there a URL for a video file/stream that you can share.. so I can reproduce your issue?

kevin4dhd commented 3 years ago

Yes, here I leave you a url of a movie, if you put the user-agent that I have shown you it works http://45.35.49.94:25461/movie/5Ip37HpXoc/j1xKakGiX4/1127.mp4

warren-bank commented 3 years ago

Looks like your assertion about User-Agent is right:

curl -I "http://45.35.49.94:25461/movie/5Ip37HpXoc/j1xKakGiX4/1127.mp4"
  # HTTP/1.1 200 OK
  # Server: nginx
  # Content-Type: video/mp4
  # Content-Length: 0

curl -I -H "User-Agent: Lavf/57.83.100" "http://45.35.49.94:25461/movie/5Ip37HpXoc/j1xKakGiX4/1127.mp4"
  # HTTP/1.1 200 OK
  # Server: nginx
  # Content-Type: video/mp4
  # Content-Length: 0

curl -o "video.mp4" "http://45.35.49.94:25461/movie/5Ip37HpXoc/j1xKakGiX4/1127.mp4"
  # 0 B

curl -o "video.mp4" -H "User-Agent: Lavf/57.83.100" "http://45.35.49.94:25461/movie/5Ip37HpXoc/j1xKakGiX4/1127.mp4"
  # OK, killed download after having received the start of file

curl -o "video.mp4" -H "User-Agent: Chrome/90" "http://45.35.49.94:25461/movie/5Ip37HpXoc/j1xKakGiX4/1127.mp4"
  # 0 B

curl -o "video.mp4" -H "User-Agent: Mozilla/5.0 (X11; Linux armv7l) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.0 Safari/537.36 CrKey/1.27.96538" "http://45.35.49.94:25461/movie/5Ip37HpXoc/j1xKakGiX4/1127.mp4"
  # 0 B

you could either:

  1. reconfigure the nginx video host to remove this silly requirement
  2. build a custom client to satisfy this requirement
    • for Chromecast: that means a custom receiver that runs on the Chromecast, and then you need to cast from a sender that uses the ID of your new receiver
    • for any generic video player: that means that you need the ability to configure the User-Agent request header

In any case, the WebCast app won't help you to achieve either goal. If you do end up building a custom Chromecast receiver app and want to build your own sender app for Android.. though you could change a few hard-coded values and build a modified binary.. there are other apps that would server as a better starting point; the most obvious one is the official ExoPlayer cast sender demo.

PS: since your video host reports a Content-Length of 0 even when the headers are properly configured.. some video players (possibly including Chromecast) may not work.

kevin4dhd commented 3 years ago

I've gotten my own local proxy to work by modifying this repository: https://github.com/giuliohome/WifiHotspotProxy Just modify this part and now it plays the mp4 in the browser, now I will send this url to the cast, I will inform you if it works

image

image

warren-bank commented 3 years ago

that's a nifty little app.. if it had a setting page to configure which headers it will dynamically modify, it could be a useful utility to keep onhand.

PS: the line of code that you added is a good test.. but you should probably also add User-Agent to REMOVE_REQUEST_HEADERS.. so you don't send 2x different values in each request.. which isn't an ideal test case.

correction:

nevermind.. setRequestProperty will add or replace an existing key.. so your code is good

kevin4dhd commented 3 years ago

something curious is that instead of adding a repeat, what it does is replace it, I think because it is a hashmap, the key remains the same but the content changes so there will never be two agents

There is only one problem, when you change for example from minute 2 to minute 10 and then to 25 and so randomly the video, there are times when the proxy stops and causes an inputstream error, there is still polish in that part since I think that the code is only used for static pages and not for streaming content like a 2 hour video

warren-bank commented 3 years ago

I'm guessing when you skip forward/backward in a static mp4.. the player will use Byte-Range headers to request that the server begin its data transfer response from a particular byte offset within the file. This proxy should forward that header to the backend server.. so I'm guessing whatever this error is.. it should probably be easy to fix (ie: trying to read from a closed connection, or some such).

warren-bank commented 3 years ago

my guess is here..

  input = connection.getInputStream();
  byte[] buffer = new byte[1024 * 32];
  int count = 0, n= 0;
  while (-1 != (n = input.read(buffer))) {
      //String finalString = new String(buffer);
      //Log.e(TAG, "path output: " + finalString);
      if (n > 0) {
          output.write(buffer, 0, n);
          output.flush();
      }
      count += n;
  }
  output.flush();
  output.close();

a possible fix might be:

  input = connection.getInputStream();
  byte[] buffer = new byte[1024 * 32];
  int count = 0, n= 0;
  while (-1 != (n = input.read(buffer))) {
    try {
      //String finalString = new String(buffer);
      //Log.e(TAG, "path output: " + finalString);
      if (n > 0) {
          output.write(buffer, 0, n);
          output.flush();
      }
      count += n;
    }
    catch(IOException e) {
      Log.e(TAG, "Connection closed by client before transfer of proxied response is complete", e);
      break;
    }
  }
  output.flush();
  output.close();
  input.close();

..which would catch attempts to write to output after it has already been closed.

kevin4dhd commented 3 years ago

I have managed to modify the code a little and it is better, the error no longer occurs when you randomly change the video range, there is still a need to improve and add an interface to add the headers dynamically, but for something I start, thank you very much for answering my questions , I have achieved my goal https://github.com/kevin4dhd/Android-Proxy-Cast

warren-bank commented 3 years ago

I just happened across another app that runs a local proxy server on Android, which gives a TON of advanced options to block or modify requests. It doesn't use any support libraries, which keeps its size down to about 100 KB. I quickly configured it to add a new header to all requests, and configured Opera Classic (opera:config) to proxy all requests through 127.0.0.1:8000.. works perfectly.

https://play.google.com/store/apps/details?id=KRIS.android.FilterProxy

https://sites.google.com/site/neutraltao/filterproxy/filterproxy-ver2-english