Trying to add a new plugin: Urplay.pm

[GoogleCodeExporter]

What steps will reproduce the problem?
1. to the best of my knowledge followed the steps here: 
http://code.google.com/p/get-flash-videos/wiki/AddingSite
2. created lib/FlashVideo/Site/Urplay.pm
3. made a deb-package and installed

What is the expected output? What do you see instead?

Didn't expect this:

Trying to open plugin ~/.get_flash_videos/plugins/Urplay.pm
Using method 'urplay' for http://urplay.se/160858
Error: Undefined subroutine &FlashVideo::Site::Urplay::find_video called at 
/usr/bin/get_flash_videos line 327.

The module file is installed: /usr/share/perl5/FlashVideo/Site/Urplay.pm

What version of the product are you using? On what operating system?

using git clone, OS debian sid.

Please provide any additional information below.

Attached module source.

Original issue reported on code.google.com by Cristian...@gmail.com on 18 Sep 2011 at 5:51

Attachments:

• Urplay.pm

[GoogleCodeExporter]

Other (unrelated?) oddities showed up:

* DOS line endings in module Canalvie.pm

* missing '*debug = \&FlashVideo::Utils::debug;' in module Canalvie.pm, 
generating errors like:

String found where operator expected at .../combined-get_flash_videos line 
8508, near "debug "VIDEOID = ""
        (Do you need to predeclare debug?)
String found where operator expected at .../combined-get_flash_videos line 
8514, near "debug "TITLE = ""
        (Do you need to predeclare debug?)
String found where operator expected at .../combined-get_flash_videos line 
8526, near "debug "URL = ""
        (Do you need to predeclare debug?)

This works around that:

--- combined-get_flash_videos.~1~       2011-09-18 23:17:12.059508998 +0200
+++ combined-get_flash_videos   2011-09-18 23:44:03.812635036 +0200
@@ -8490,7 +8490,7 @@ package FlashVideo::Site::Canalvie;

 use strict;
 BEGIN { FlashVideo::Utils->import(); } # (added by utils/combine-perl.pl)
-BEGIN { no strict 'refs';  *title_to_filename = 
\&FlashVideo::Utils::title_to_filename; *from_xml = 
\&FlashVideo::Utils::from_xml; }
+BEGIN { no strict 'refs'; *debug = \&FlashVideo::Utils::debug; 
*title_to_filename = \&FlashVideo::Utils::title_to_filename; *from_xml = 
\&FlashVideo::Utils::from_xml; }

 sub find_video {
   my($self, $browser, $embed_url, $prefs) = @_;

Original comment by Cristian...@gmail.com on 18 Sep 2011 at 10:14

[GoogleCodeExporter]

Alright.  I managed to get something working.  An Urlplay.pm module (attached). 
 urplay.se is a swedish documentary channel.  Can download subtitles too.  I'm 
sure it can be improved.  The only trouble I have with it is not finding 
out/managing how to get a non-utf8 encoded file name out of it.  Subtitles are 
xml-files.  'subtitleeditor' can convert them to .srt files.  'recode' can 
convert 'utf8..latin1' when 'subtitleeditor' can't.

Original comment by Cristian...@gmail.com on 19 Sep 2011 at 8:14

Attachments:

• Urplay.pm

[GoogleCodeExporter]

Sorry for the slow response. A few comments:

- Don't use bareword/global filehandles: 
https://www.socialtext.net/perl5/bareword_uppercase_filehandles
- You're appending to the subtitles file which is probably not correct. (For 
example, if someone tries to download the same video twice, the subtitles will 
be written to the file twice.)
- Please try to convert the subtitles to SRT/SubRip. The 
convert_smi_subtitles_to_srt function in FlashVideo/Utils.pm could be useful.
- I don't quite understand your comment about non-UTF8 encoded filenames. Do 
you actually want the filename to be in UTF8? Filesystems are generally a mess 
for internationalisation. If you want to use a different encoding, look at the 
Perl Encode module (core). Also consider using this module for changing the 
encoding of subtitles. It's pretty simple.
-  if ($prefs->{subtitles} == 1) { - there is no need for the "== 1" comparison.
-       $browser->get("$sub"); - there is no need to quote $sub
- binmode TT, ':utf8' is probably not what you want. It just blindly trusts 
that the data is UTF8. You probably want ':encoding(UTF-8)', but read perldoc 
perluniintro.
- I'm possibly missing something but returning a list of rtmpdump commands 
seems broken to me. Return a hash reference instead. get_flash_videos will then 
construct the rtmpdump command.

Original comment by zakflash...@gmail.com on 9 Apr 2012 at 3:59

[GoogleCodeExporter]

I corrected a few things you pointed out.

Tried to figure out how to convert subtitles to subrip-format, but no joy.  As 
I mentioned, the subtitle file served is "Timed Text Authoring Format 1.0", as 
reported by subtitleeditor (program I subsequently use to convert to SubRip 
format).

The "broken" list of returned rtmpdump commands is copied from 
Svtplay.pm/Tv3play.pm.

Original comment by Cristian...@gmail.com on 17 May 2012 at 10:34

Attachments:

• Urplay.pm

[GoogleCodeExporter]

Glad to see what you're making progress. A few more comments:

- Don't use Text::Iconv. Use Encode.
- Make your debug output a bit more helpful. "content1" or "url2" are not very 
useful.
- () are not needed around open, binmode, print and close in your code.
- $args is not a good name for the data structure containing RTMP parameters. 
Why not call it $rtmpdump_args or $video_details or something like that?
- Don't quote variables in $args. In other words, do rtmp => $url, playpath => 
$playpath etc.
- I mentioned the convert_smi_subtitles_to_srt function because it might have 
been useful--in other words, Urplay might have used SMI subtitles. However, as 
you point out, they use timed text instead. But looking at a sample file 
(http://undertexter.ur.se/167000-167999/167789-14.tt) I think it's simple to 
convert this to SubRip. See my attached code for an example, though note that I 
haven't tested whether the subtitles actually sync correctly. Please try it and 
let me know. Note it outputs subtitles in UTF-8. This may or may not be what 
you want. Some media players seem to assume subtitle files are in ISO-8859-1 
(Latin1). With Encode it is easy to change this though.
- You still use the pointless == 1) when checking if subtitles are enabled.
- Please add a test case (see t/urls).

Original comment by zakflash...@gmail.com on 20 May 2012 at 7:34

Attachments:

• tt2srt.pl

[GoogleCodeExporter]

Thanks for the comments and code.  This is another attempt to make something 
sensible of Urplay.pm.

* replaced use of Text::Iconv with Encode
* removed useless debug code
* removed () around open, binmode, print and close
* changed $args to $rtmpdump_args
* no quotes around variables in $rtmpdump_args
* included most of your code in Urplay.pm; subtitles in UTF-8 are fine and the 
subtitles actually sync correctly
* removed pointless '== 1)' when checking if subtitles are enabled
* adding a test case: not done yet, but urls are available for a limited time 
only as flash videos are periodically removed, thus tests will eventually 
break; though, this <http://urplay.se/166599> could be one example

OT: just noticed the video file names downloaded with Svtplay.pm are a (UTF-8) 
mess :(  one can hardly TAB-complete in order to rename them.

Original comment by Cristian...@gmail.com on 23 May 2012 at 8:29

Attachments:

• Urplay.pm

[GoogleCodeExporter]

Almost there, thanks for these changes. Just one point:

Don't save the TT subtitles to a file. Just keep them in memory, and change the 
convert_to_srt function to accept the subtitles text. This is slightly faster 
and allows the "slurp" function to be removed. (I know my code read the file 
off disk, but this was just to make things quicker during testing, rather than 
downloading the file over and over.)

As for Svtplay.pm, it sounds like the filenames are possibly double-encoded or 
otherwise corrupt. Please provide a URL and I will have a look.

Original comment by zakflash...@gmail.com on 27 May 2012 at 4:46

[GoogleCodeExporter]

Svtplay.pm: here's an example 
<http://svtplay.se/v/2800505/varldens_mysterier/12_5?cb,a1364145,1,f,-1/pb,a1364
142,1,f,-1/pl,v,,2800505/sb,p178188,1,f,-1>.
The name of the downloaded file should be either 
12_5_-_V_rldens_mysterier___SVT_Play.flv or 
12_5_-_Världens_mysterier___SVT_Play.flv but not 
12_5_-_Världens_mysterier___SVT_Play.flv.

I'll have a look at code and try to skip storing the TT subtitles to a file.

Original comment by Cristian...@gmail.com on 27 May 2012 at 5:11

[GoogleCodeExporter]

Alright.  No TT subtitles file saved, just SRT.

Though I noticed some odd 'title_to_filename' function behaviour; example url 
<http://urplay.se/144643>.  If the title starts with an '_' (underscore), in 
this example '_sterrike', the function 'title_to_filename' removes the '_' 
making the SRT file name 'sterrike.srt' instead of the expected '_sterrike.srt'.

Original comment by Cristian...@gmail.com on 27 May 2012 at 5:57

Attachments:

• Urplay.pm

[GoogleCodeExporter]

Bummer :(  Urplay.pm is broken.  Something changed.  Can anyone make sense 
(rtmpdump command line?) out of the attached html-file fetched from 
<http://urplay.se/170556>?

Cheers,

-- 
Cristian

Original comment by Cristian...@gmail.com on 10 Jul 2012 at 10:20

Attachments:

• pappersl_sa-flyktingar.html.gz

[GoogleCodeExporter]

Finally figured it out.  Problem was find_video returning a list instead of a 
hash.  Changing that keeps me going a while longer ;)  Code attached.

Cheers,

-- 
Cristian

Original comment by Cristian...@gmail.com on 15 Jul 2012 at 9:29

Attachments:

• Urplay.pm