Closed GoogleCodeExporter closed 9 years ago
I've been experiencing the same behavior and did some investigation. I found
that for my subscriptions youtube has added more formats recently than I've had
in the past. There is a format map variable in the javascript code that breaks
out to something like this:
37/1920x1080/9/0/115,
22/1280x720/9/0/115,
35/854x480/9/0/115,
34/640x360/9/0/115,
18/640x360/9/0/115,
5/320x240/7/0/0
The first value is the id (some of these are used by rsshandler currently).
Now the rub comes in videoservlet.java. It runs a regex across the entire page
for the video to try to find the binary you desire. This is a group match and
it grabs the first result it finds on the page. I'm not sure how this worked
with youtube previously, but whatever they changed now makes it select a much
higher resolution (depending on what is available).
I pulled down the code and made a small POC hack videoservlet.java to see if I
could get a lower resolution (and thus file size). I found that the ids in the
format map appear to map to a url parameter named itag that getVideoLink2
returns. I changed getVideoLink2 to the following:
private String getVideoLink2(String content, int fmt) throws
UnsupportedEncodingException {
Pattern pattern = Pattern.compile("%7C(http.+?videoplayback.+?)%2C");
Matcher matcher = pattern.matcher(content);
String format = "%26itag%3D" + fmt + "%26";
String url = null;
while (matcher.find()) {
String u = matcher.group(1);
if (u.indexOf(format) > -1) {
logger.info("getVideoLink2: match (" + u + ") for format " + fmt);
url = u;
break;
}
}
if (url != null) {
return URLDecoder.decode(url, "UTF-8");
}
throw new IllegalStateException("Cann't find video file address: "+ content);
}
The call in doGet needs a slight modification as well:
String redirect = getVideoLink2(str,format);
This by no means complete, but thought I would share my analysis so far.
Original comment by jeromeku...@gmail.com
on 13 Dec 2010 at 6:33
jorome: your work is much appreciated. I'm simply a user and not a Java
programmer, but would like to see this fixed asap as rsshandler has become
useless for me to download and sync YouTube channels to my phone for watching
on flights.
Original comment by david%da...@gtempaccount.com
on 13 Dec 2010 at 8:47
thanks mr. jerome dude, your effort is highly appreciated.
since you're a tru blu java guy, i wanted to ask a few quick questions:
1. while we're in videoserevlet.java,
how can you set a cookie (using the youtube login) in the doGet?
i did this:
URLConnection connection = url.openConnection();
String myCookie = "LOGIN_INFO=<blah blah blah cookie data from browser blah blah blah>";
connection.setRequestProperty("Cookie", myCookie);
it works, but it looks so ugly and the cookie is set to expire in 6 months (i
think). that means i have to rebuild it every 6 months.
this is so that we'd be able to get those restricted videos.
2. how do you build a signed jar? moreover how do you build it using gradle?
running on win32 here :) i was able to build the jar but its unsigned so i have
to activate it always via command line.
Thanks again!:)
Original comment by wildkard...@gmail.com
on 14 Dec 2010 at 7:00
Original comment by jeromeku...@gmail.com
on 16 Dec 2010 at 3:23
here's a good list of codes, codecs and formats:
https://secure.wikimedia.org/wikipedia/en/wiki/YouTube#Quality_and_codecs
haven't tried format 45 (webm hd) yet, mirroring my first channel in regular hd!
Original comment by LKooglizmus
on 16 Dec 2010 at 9:02
youtube removed the fmt parameters since start of december, now there is only
the hd=1 option. Can you see if you can make the rsshandler work with the new
youtube format modifications.
Thanks
Original comment by EdO...@gmail.com
on 16 Dec 2010 at 3:02
[deleted comment]
i'm downloading with only format=22 in the URL, and i'm getting HD content
where it exists.. with only hd=1, how would they differentiate between
requesting 720p or 1080p?
Original comment by LKooglizmus
on 16 Dec 2010 at 3:07
Original comment by jeromeku...@gmail.com
on 18 Dec 2010 at 7:56
Why is this issue set to "fixed", how is it fixed, and how can I get the fix?
Original comment by spammerh...@gmail.com
on 21 Dec 2010 at 3:43
Fixed means developer has made source changes and QA still needs to be done to
verify. You can sync the code from the repository and build locally to help
with the testing effort.
http://code.google.com/p/rsshandler/source/checkout
Original comment by jeromeku...@gmail.com
on 22 Dec 2010 at 5:14
^How do we do this exactly?
Original comment by Jorge.Me...@gmail.com
on 23 Dec 2010 at 2:13
Jorge,
It will require setting up the build environment, syncing the code tree, compiling and running the build you just created. At this time I'm not prepared to release a binary as I'm still QAing it myself. I started a build environment wiki page to further help out.
http://code.google.com/p/rsshandler/wiki/BuildEnvironment
Original comment by jeromeku...@gmail.com
on 23 Dec 2010 at 6:25
Well, in any case, thanks for your hard work. I appreciate it.
Original comment by Jorge.Me...@gmail.com
on 23 Dec 2010 at 10:15
thanks for the hard work ... hope we get a new build soon
Original comment by ilan...@gmail.com
on 29 Dec 2010 at 6:54
Yes, hope it get fixed, thanks again..
Original comment by Hansi...@gmail.com
on 29 Dec 2010 at 11:33
A beta build is posted that you can all try out that should resolve this bug.
http://code.google.com/p/rsshandler/downloads/detail?name=rsshandler_3.0.1_beta.
zip&can=2&q=
If you use the JNLP file, then you will not get this build at this time.
Original comment by jeromeku...@gmail.com
on 9 Jan 2011 at 7:46
Thank you so much! It's working well for me so far.
Original comment by Jorge.Me...@gmail.com
on 9 Jan 2011 at 5:00
Original issue reported on code.google.com by
Jorge.Me...@gmail.com
on 8 Dec 2010 at 3:14