xiel / rsshandler

Automatically exported from code.google.com/p/rsshandler
1 stars 0 forks source link

Downloads wrong video format #27

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
I'm using version 2.0 on Windows XP SP3.

As of 2 days ago, the rsshandler program has been downloading the 1280x720 
version of a video instead of the 640x360 version it usually gets since I set 
it to output "MP4 (iTunes)", not "HD (MP4)".

Any help would be appreciated.

Original issue reported on code.google.com by Jorge.Me...@gmail.com on 8 Dec 2010 at 3:14

GoogleCodeExporter commented 9 years ago
I've been experiencing the same behavior and did some investigation.  I found 
that for my subscriptions youtube has added more formats recently than I've had 
in the past.  There is a format map variable in the javascript code that breaks 
out to something like this:

37/1920x1080/9/0/115,
22/1280x720/9/0/115,
35/854x480/9/0/115,
34/640x360/9/0/115,
18/640x360/9/0/115,
5/320x240/7/0/0

The first value is the id (some of these are used by rsshandler currently).  
Now the rub comes in videoservlet.java.  It runs a regex across the entire page 
for the video to try to find the binary you desire.  This is a group match and 
it grabs the first result it finds on the page.  I'm not sure how this worked 
with youtube previously, but whatever they changed now makes it select a much 
higher resolution (depending on what is available).

I pulled down the code and made a small POC hack videoservlet.java to see if I 
could get a lower resolution (and thus file size). I found that the ids in the 
format map appear to map to a url parameter named itag that getVideoLink2 
returns. I changed getVideoLink2 to the following:

private String getVideoLink2(String content, int fmt) throws 
UnsupportedEncodingException {
      Pattern pattern = Pattern.compile("%7C(http.+?videoplayback.+?)%2C");
      Matcher matcher = pattern.matcher(content);
      String format = "%26itag%3D" + fmt + "%26";
      String url = null;

      while (matcher.find()) {
        String u = matcher.group(1);

        if (u.indexOf(format) > -1) {
            logger.info("getVideoLink2: match (" + u + ") for format " + fmt);
            url = u;
            break;
        }
      }

      if (url != null) {
        return URLDecoder.decode(url, "UTF-8");
      }
      throw new IllegalStateException("Cann't find video file address: "+ content);
}

The call in doGet needs a slight modification as well:

String redirect = getVideoLink2(str,format);

This by no means complete, but thought I would share my analysis so far.

Original comment by jeromeku...@gmail.com on 13 Dec 2010 at 6:33

GoogleCodeExporter commented 9 years ago
jorome: your work is much appreciated. I'm simply a user and not a Java 
programmer, but would like to see this fixed asap as rsshandler has become 
useless for me to download and sync YouTube channels to my phone for watching 
on flights.

Original comment by david%da...@gtempaccount.com on 13 Dec 2010 at 8:47

GoogleCodeExporter commented 9 years ago
thanks mr. jerome dude, your effort is highly appreciated.

since you're a tru blu java guy, i wanted to ask a few quick questions:

1. while we're in videoserevlet.java,
how can you set a cookie (using the youtube login) in the doGet?

i did this:
    URLConnection connection = url.openConnection();
    String myCookie = "LOGIN_INFO=<blah blah blah cookie data from browser blah blah blah>";
    connection.setRequestProperty("Cookie", myCookie);

it works, but it looks so ugly and the cookie is set to expire in 6 months (i 
think). that means i have to rebuild it every 6 months.

this is so that we'd be able to get those restricted videos.

2. how do you build a signed jar? moreover how do you build it using gradle? 
running on win32 here :) i was able to build the jar but its unsigned so i have 
to activate it always via command line.

Thanks again!:)

Original comment by wildkard...@gmail.com on 14 Dec 2010 at 7:00

GoogleCodeExporter commented 9 years ago

Original comment by jeromeku...@gmail.com on 16 Dec 2010 at 3:23

GoogleCodeExporter commented 9 years ago
here's a good list of codes, codecs and formats: 
https://secure.wikimedia.org/wikipedia/en/wiki/YouTube#Quality_and_codecs

haven't tried format 45 (webm hd) yet, mirroring my first channel in regular hd!

Original comment by LKooglizmus on 16 Dec 2010 at 9:02

GoogleCodeExporter commented 9 years ago
youtube removed the fmt parameters since start of december, now there is only 
the hd=1 option. Can you see if you can make the rsshandler work with the new 
youtube format modifications.
Thanks

Original comment by EdO...@gmail.com on 16 Dec 2010 at 3:02

GoogleCodeExporter commented 9 years ago
[deleted comment]
GoogleCodeExporter commented 9 years ago
i'm downloading with only format=22 in the URL, and i'm getting HD content 
where it exists.. with only hd=1, how would they differentiate between 
requesting 720p or 1080p?

Original comment by LKooglizmus on 16 Dec 2010 at 3:07

GoogleCodeExporter commented 9 years ago

Original comment by jeromeku...@gmail.com on 18 Dec 2010 at 7:56

GoogleCodeExporter commented 9 years ago
Why is this issue set to "fixed", how is it fixed, and how can I get the fix?

Original comment by spammerh...@gmail.com on 21 Dec 2010 at 3:43

GoogleCodeExporter commented 9 years ago
Fixed means developer has made source changes and QA still needs to be done to 
verify.  You can sync the code from the repository and build locally to help 
with the testing effort.

http://code.google.com/p/rsshandler/source/checkout 

Original comment by jeromeku...@gmail.com on 22 Dec 2010 at 5:14

GoogleCodeExporter commented 9 years ago
^How do we do this exactly?

Original comment by Jorge.Me...@gmail.com on 23 Dec 2010 at 2:13

GoogleCodeExporter commented 9 years ago
Jorge,
    It will require setting up the build environment, syncing the code tree, compiling and running the build you just created.  At this time I'm not prepared to release a binary as I'm still QAing it myself.  I started a build environment wiki page to further help out.

http://code.google.com/p/rsshandler/wiki/BuildEnvironment

Original comment by jeromeku...@gmail.com on 23 Dec 2010 at 6:25

GoogleCodeExporter commented 9 years ago
Well, in any case, thanks for your hard work. I appreciate it.

Original comment by Jorge.Me...@gmail.com on 23 Dec 2010 at 10:15

GoogleCodeExporter commented 9 years ago
thanks for the hard work ... hope we get a new build soon

Original comment by ilan...@gmail.com on 29 Dec 2010 at 6:54

GoogleCodeExporter commented 9 years ago
Yes, hope it get fixed, thanks again..

Original comment by Hansi...@gmail.com on 29 Dec 2010 at 11:33

GoogleCodeExporter commented 9 years ago
A beta build is posted that you can all try out that should resolve this bug.

http://code.google.com/p/rsshandler/downloads/detail?name=rsshandler_3.0.1_beta.
zip&can=2&q=

If you use the JNLP file, then you will not get this build at this time. 

Original comment by jeromeku...@gmail.com on 9 Jan 2011 at 7:46

GoogleCodeExporter commented 9 years ago
Thank you so much! It's working well for me so far.

Original comment by Jorge.Me...@gmail.com on 9 Jan 2011 at 5:00