search

giavoni / get-flash-videos

Automatically exported from code.google.com/p/get-flash-videos
0 stars 0 forks source link

Trying to add support for www.nasa.gov/multimedia/videogallery but it does agent string checking #367

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?

1. Added Nasa.pm to Sites using Example.pm as per the wiki/AddingSite

2. ran get_flash_videos
$ get_flash_videos -d 
http://www.nasa.gov/multimedia/videogallery/index.html?media_id=135251341
No plugins installed
Trying to open plugin ~/.get_flash_videos/plugins/Www.pm
Using method 'nasa' for 
http://www.nasa.gov/multimedia/videogallery/index.html?media_id=135251341
Downloading 
http://www.nasa.gov/multimedia/videogallery/index.html?media_id=135251341
-> GET http://www.nasa.gov/multimedia/videogallery/index.html?media_id=135251341
<- 200 text/html (22769): UTF8 on, non-ASCII, 22769 characters 22773 bytes
Error: 
Couldn't extract Flash movie URL. This site may need specific support adding,
or fixing.

Please confirm the site is using Flash video and if you have Flash available
check that the URL really works(!).

Check for updates by running: Download/get-flash-videos/get_flash_videos-1.25 
--update

If the latest version does not support this please open a bug (or
contribute a patch!) at http://code.google.com/p/get-flash-videos/
make sure you include the output with --debug enabled.
Couldn't download any videos.

2.$browser->content variable in Nasa.pm does not have any relevant information
3.$browser->content contains the following:

<div id="video_player_holder" class="video_player">
    <div id="video_player">
    <!--<b>You do not seem to have Adobe Flash Player 10 required to view this video. Please download the player <a href="http://www.adobe.com/go/getflashplayer">here</a></b>-->
    </div>
</div>

What is the expected output? What do you see instead?
$browser->content with some links I can follow

What version of the product are you using? On what operating system?
Latest git. Fedora 14

Please provide any additional information below.

I am guessing that the site examines the user agent string. AFAICT it does only 
one GET before getting the deficient response. Is there some universal cookie 
that flashplayer can return to a site? So I am guessing it is the user agent 
string that causes the site to respond in this manner.

Is there a way to set the agent string to spoof it into thinking the GET is 
from Firefox?

Original issue reported on code.google.com by pcwal...@comcast.net on 11 Mar 2012 at 9:41

GoogleCodeExporter commented 9 years ago 
A useful tool for adding support for sites to get_flash_videos is Burp 
(http://www.portswigger.net/burp/download.html). This is a proxy server which 
allows you to log and inspect HTTP traffic. It also allows you to select 
requests and send them to the built-in "repeater" tool. This allows you to 
change bits of the request and retry it, and see if you still get the same 
results. For example, you could try removing the user-agent, repeating the 
requesting and seeing if it still works. It's a very useful tool for 
experimentation and investigation.

It's very easy to set the user-agent to whatever you want, simply do:

  $browser->add_header('User-Agent' => $whatever);

Keep trying.

Original comment by zakflash...@gmail.com on 14 Mar 2012 at 9:30

GoogleCodeExporter commented 9 years ago 
Thanks for the encouragement.

I have cracked the nut on this site. Turns out that it uses javascript to 
rewrite the html on the browser. So if you just get the html, then there is no 
video information. I used the firebug add-on in firefox to track down the 
javascript. Wireshark also helped identify the final GET for the video file 
name. Ultimately I had to GET the javascript, extract some variables and 
compute the the video file url like the javascript code, before I could fetch 
the video.

I'll try this code for awhile to see if there are any bugs in it before I 
submit it.

How do I submit a site to the git anyway? The wiki explains how to clone the 
git, but not how to submit a patch or add?

Original comment by pcwal...@comcast.net on 21 Mar 2012 at 4:21

GoogleCodeExporter commented 9 years ago 
Good work.

Fork get-flash-videos on Github, and then clone your fork.

Add your file with git add and then add your changes with git commit and git 
push. Log in to Github and view your fork.

Click the "pull request" button near the top right of the window, and follow 
the instructions. We get notified and if we're happy, we can merge the pull 
request. Github also has pretty good tools for commenting and reviewing code, 
so you can refine the code if we have any concerns.

Look forward to seeing the code.

Original comment by zakflash...@gmail.com on 21 Mar 2012 at 9:59

GoogleCodeExporter commented 9 years ago 
I worked through the process of forking get-flash-videos and added Nasa.pm.

I have committed and pushed a few Ustream.pm changes as well as the new Nasa.pm.

I submitted a pull request. However, I am not sure what to do next.

Original comment by pcwal...@comcast.net on 26 Mar 2012 at 6:37

GoogleCodeExporter commented 9 years ago 
Just merged your changes, thanks. Once you submit a pull request, you just have 
to wait for it to be merged. We're not as quick as we should be, so thanks for 
your patience.

Original comment by zakflash...@gmail.com on 9 Apr 2012 at 3:13