search

gco / rubyripper

Automatically exported from code.google.com/p/rubyripper
0 stars 0 forks source link

crash while fetching freedb info with artist in UTF-8 #428

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. insert a cd with an artist with UTF (i.e. umlauts) on FreeDB/your cddb
2. make sure freedb.yaml and .cddb don't include the data
(otherwise the problem will only occur later on differently (see second trace))
3. rrip_cli

I get this:

Audio-CD gefunden, Anzahl der Titel: 22, Gesamtspieldauer: 39:11
Hole Freedb Daten...
preparing to contact freedb server
...
Created fetch string:
/~cddb/cddb.cgi?cmd=cddb+read+jazz+07092f16&hello=anonymous+my_secret.com+rubyri
pper+0.6.0rc1&proto=6
Raw P-W sub-channel reading (audio track) is supported.
Analyzing track 01 (AUDIO): start 00:00:00, length 03:03:42...
00:03:00
FREEDB INFO

CD INFO
/usr/bin/rrip_cli:294:in `showFreedb': incompatible character encodings:
UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
        from /usr/bin/rrip_cli:265:in `handleFreedb'
        from /usr/bin/rrip_cli:244:in `get_cd_info'
        from /usr/bin/rrip_cli:47:in `initialize'
        from /usr/bin/rrip_cli:486:in `new'
        from /usr/bin/rrip_cli:486:in `<main>'

This is this line:
puts _("Artist:") + " #{@settings['cd'].md.artist}"
This release is from the artist "Jäger 90"
I can reproduce this problem with another disc having this artist:
"L'âme immortell".
I only have this problem when the artist is in UTF-8 (non-ascii part).
Other releases with UTF release names or titles were no problem this week.

Afterwards it is still going on with disc analysis:

Found 14 Q sub-channels with CRC errors.
Control nibbles of track match CD-TOC settings.
Analyzing track 02 (AUDIO): start 03:03:42, length 03:49:45...
Found 2 Q sub-channels with CRC errors.
..
Control nibbles of track match CD-TOC settings.
Found CD-TEXT data.

Reading of toc data finished successfully

but stopps after that.

If the freedb data is already in freedb.yaml then it runs further through
the procedure, but is not creating any files and finally stops with:
Ripping progress (100 %)
/usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2411:in `mp3': incompatible
character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
        from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2281:in `doMp3'
        from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2178:in `encodeTrack'
        from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2140:in `block (2
levels) in startEncoding'

This will probably be fixed with the above mentioned problem, but I wanted
to note it.

I attach the freedb information that is saved in freedb.yaml for this
release. I also attach the complete log of the first problem, but I think I
stated everything important.

Original issue reported on code.google.com by goo...@JonnyJD.net on 31 May 2010 at 5:38

Attachments:

GoogleCodeExporter commented 9 years ago 
That other "L'âme immortell" release was working somewhen, because I 
successfully
ripped it before. The files have creation times in December 2009, but I can't 
tell
anymore what version of rubyripper I used for that.

Original comment by goo...@JonnyJD.net on 31 May 2010 at 5:43

GoogleCodeExporter commented 9 years ago 
Fix for the first error:
http://github.com/rubyripperdev/rubyripper/commit/8712c6f029a820b7d8faff4ca7ee42
d9d352
e93c

Original comment by boukewou...@gmail.com on 31 May 2010 at 6:05

GoogleCodeExporter commented 9 years ago 
Fix for the second error:
http://github.com/rubyripperdev/rubyripper/commit/aa556907fe898b23e0ba34e4695dac
af7cc
6613c

I am not 100% sure this fixes the problem. I at least prevented the '+'-sign to 
combine strings. So I hope the strings will be automatically encoded as UTF-8 
as soon 
as one element in it contains a non-ASCII character.

Ripping this disc once again and reporting the results would be much 
appreciated :)

Original comment by boukewou...@gmail.com on 31 May 2010 at 6:24

GoogleCodeExporter commented 9 years ago 
CDDB INFO
/usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2606:in `updateGui': incompatible 
character
encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2462:in `startRip'
    from /usr/bin/rrip_cli:427:in `dir_exists'
    from /usr/bin/rrip_cli:447:in `update'
    from /usr/bin/rrip_cli:414:in `prepareRip'
    from /usr/bin/rrip_cli:331:in `showFreedbOptions'
    from /usr/bin/rrip_cli:305:in `showFreedb'
    from /usr/bin/rrip_cli:265:in `handleFreedb'
    from /usr/bin/rrip_cli:244:in `get_cd_info'
    from /usr/bin/rrip_cli:47:in `initialize'
    from /usr/bin/rrip_cli:486:in `new'
    from /usr/bin/rrip_cli:486:in `<main>'

Now I get this (after box fixes)

Do you do anything different with artists then with release names?

Original comment by goo...@JonnyJD.net on 31 May 2010 at 6:33

GoogleCodeExporter commented 9 years ago 
both fixes.. I meant

Original comment by goo...@JonnyJD.net on 31 May 2010 at 6:33

GoogleCodeExporter commented 9 years ago 
Fix for the third error:
http://github.com/rubyripperdev/rubyripper/commit/48a92e371d3a1f01bdfd069d2221e2
56e32
3784e

I don't think I understand your question. It's just a problem with combining 
strings 
that are encoded differently (UTF-8 versus ASCCI).

I'll keep it open for a while just in case.

Original comment by boukewou...@gmail.com on 31 May 2010 at 6:45

GoogleCodeExporter commented 9 years ago 
Yes, but it is the same with release names (and titles). So if you do the same 
things
for these strings, there should either be no problem or problems for both.

But the UTF handling of strings in ruby seems weird to me anyways..

Right now I am ripping again and don't have problems at the moment, but no 
mp3/flac
are created either still.
Is multithreading disabled in debug mode?

Original comment by goo...@JonnyJD.net on 31 May 2010 at 7:01

GoogleCodeExporter commented 9 years ago 
and here we go again with the next trace:

Ripping progress (100 %)
/usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2428:in `+': can't convert nil into 
String
(TypeError)
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2428:in `checkCommand'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2413:in `mp3'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2281:in `doMp3'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2178:in `encodeTrack'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2140:in `block (2 levels) in startEncoding'

Original comment by goo...@JonnyJD.net on 31 May 2010 at 7:07

GoogleCodeExporter commented 9 years ago 
Fix for fourth error:
http://github.com/rubyripperdev/rubyripper/commit/40abcb8af927ed70518d2f56ec3b89
65d87
216c8

It seems that ruby-1.9 scans any new string if it contains only ASCCI 
characters, if 
so encoding is ASCCI. If it contains any UTF-8 characters it's saved as UTF-8. 
When 
joining these strings with '+' they have to be of the same type.

Lucky for me, the other notation with the #{variable} seems to work in strings, 
even 
if the #{variable} is of an UTF-8 type and the rest is ASCCI.

Keep on going, it must end somewhere, right?

Original comment by boukewou...@gmail.com on 31 May 2010 at 7:18

GoogleCodeExporter commented 9 years ago 
I've just committed a fix that may help discovering more bugs:
http://github.com/rubyripperdev/rubyripper/commit/5c44b73d23bfe8ff8a3b47a08ea41f
1be324
9402

Original comment by boukewou...@gmail.com on 31 May 2010 at 7:25

GoogleCodeExporter commented 9 years ago 
Okay, I will test that. Before that I have this trace now:

(note: I was only ripping the first 3 files this time)

Ripping progress (100 %)
Adding track 3 (mp3) to the queue..Adding track 3 (flac) to the queue..

/usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2150:in `join': deadlock detected 
(fatal)
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2150:in `block in startEncoding'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2150:in `each'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2150:in `startEncoding'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2124:in `addTrack'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:1824:in `ripTrack'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:1794:in `block in ripTracks'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:1790:in `each'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:1790:in `ripTracks'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:1784:in `initialize'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2471:in `new'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2471:in `startRip'
    from /usr/bin/rrip_cli:427:in `dir_exists'
    from /usr/bin/rrip_cli:447:in `update'
    from /usr/bin/rrip_cli:414:in `prepareRip'
    from /usr/bin/rrip_cli:331:in `showFreedbOptions'
    from /usr/bin/rrip_cli:305:in `showFreedb'
    from /usr/bin/rrip_cli:265:in `handleFreedb'
    from /usr/bin/rrip_cli:244:in `get_cd_info'
    from /usr/bin/rrip_cli:47:in `initialize'
    from /usr/bin/rrip_cli:486:in `new'
    from /usr/bin/rrip_cli:486:in `<main>'

Original comment by goo...@JonnyJD.net on 31 May 2010 at 7:28

GoogleCodeExporter commented 9 years ago 
To me it looks like the last stacktrace suppresses stack traces.

I tried to rip the first tracks with it and rubyripper runs through.
However, I get errors and no mp3 files:

WARNUNG: Kodierung zu mp3 mit Fehler in Track 1 beendet!
Encoding progress (22.2 %)
Removing track 1 (mp3) from the queue..
...
track2_1.wav: Verify OK, wrote 25365320 bytes, ratio=0,626
Encoding progress (72.2 %)
Removing track 2 (flac) from the queue..
command = 
nice: Mit einer Priorität muss ein Befehl angegeben werden
„nice --help“ gibt weitere Informationen.
WARNUNG: Kodierung zu mp3 mit Fehler in Track 2 beendet!
Encoding progress (100 %)
Removing track 2 (mp3) from the queue..
Inside the finished function
Encoding progress (100 %)

Sorry, some error messages are german:
WARNUNG: Kodierung zu mp3 mit Fehler in Track 1 beendet!
=
WARNING: encoding to mp3 finished with errors in track 1!

command = 
nice: Mit einer Priorität muss ein Befehl angegeben werden
„nice --help“ gibt weitere Informationen.
=
This means that the program "nice" didn't get any command.

I will start another run with LC_ALL=C, but maybe this already helps.

Original comment by goo...@JonnyJD.net on 31 May 2010 at 7:44

GoogleCodeExporter commented 9 years ago 
argh.. what I meant to say in the first sentence:
"the last commit supresses stack traces"

Original comment by goo...@JonnyJD.net on 31 May 2010 at 7:45

GoogleCodeExporter commented 9 years ago 
Okay, here is the english version of the error messages. As noted, I don't get 
stack
traces anymore, but I also don't get any mp3 files anymore. The two flac files 
are
created.
I have debug and verbose mode enabled.

Ripping progress (100 %)
Adding track 2 (mp3) to the queue..Adding track 2 (flac) to the queue..

command = 
nice: a command must be given with an adjustment
Try `nice --help' for more information.
WARNING: Encoding to mp3 exited with an error with track 2!
Encoding progress (72.2 %)
Removing track 2 (mp3) from the queue..

I attach the complete log (trying to rip the first 2 tracks to flac and mp3)

I didn't try the last commit yet (1edecdc39c821cde48b9).
I will try that now.

Original comment by goo...@JonnyJD.net on 31 May 2010 at 8:30

Attachments:

GoogleCodeExporter commented 9 years ago 
This is a totally different bug, thread related. I've tried to fix this with 
commit:
http://github.com/rubyripperdev/rubyripper/commit/1edecdc39c821cde48b9ee6ad0d6bf
245b6a
e006

I am not really sure why this problem is there, but I'll change my code to be 
on the 
safe side. Only drawback is that if the threads are all used, and a new track 
is 
offered, the ripping has to wait until this tracks get started.

Original comment by boukewou...@gmail.com on 31 May 2010 at 8:31

GoogleCodeExporter commented 9 years ago 
Okay, and another fix for the different codecs:
http://github.com/rubyripperdev/rubyripper/commit/ac30a1c90739cb17c0e51a873b0cf1
9f15c3
e140

Well, let me know.. By the way, I'm really glad this is detected before the 
actual 
release :D

Original comment by boukewou...@gmail.com on 31 May 2010 at 9:03

GoogleCodeExporter commented 9 years ago 
Okay, I started another issue, because this has itself nothing to do with UTF 
anymore
(but with the fixes from this issue):

http://code.google.com/p/rubyripper/issues/detail?id=429

Original comment by goo...@JonnyJD.net on 31 May 2010 at 9:04

GoogleCodeExporter commented 9 years ago 
Well, this issue certainly served it's goal. Case closed.

Original comment by boukewou...@gmail.com on 31 May 2010 at 9:17

GoogleCodeExporter commented 9 years ago 
We have to reopen this. This was not fixed, but hidden by issue 429

The problem is:

/usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2417:in `mp3': incompatible character
encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2283:in `doMp3'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2180:in `encodeTrack'
    from /usr/lib/ruby/site_ruby/1.9.1/rr_lib.rb:2142:in `block (2 levels) in startEncoding'

Yes, this is exactly the first trace mentioned.

http://github.com/rubyripperdev/rubyripper/commit/aa556907fe898b23e0ba34e4695dac
af7cc6613c
did not help in general.
Doing this in a way without breaking issue 429 would be:

                command = String.new.force_encoding("ASCII-8BIT")
                command = "#{command}lame #{@settings['mp3settings']} #{tags}\"\
#{@out.getTempFile(track, 1)}\" \"#{filename}\""
                command = "#{command} 2>&1" unless @settings['verbose']

However, this also doesn't help. I still get the same crash. So we can safely 
fix
issue 429 the easy way (revert one commit and another one partially) and turn 
to this
issue afterwards.

Original comment by goo...@JonnyJD.net on 4 Jun 2010 at 8:31

GoogleCodeExporter commented 9 years ago

Original comment by boukewou...@gmail.com on 4 Jun 2010 at 9:32

GoogleCodeExporter commented 9 years ago 
The temporary filename only includes ASCII characters but we still have to 
force the
encoding to be ASCII-8BIT for lame.

this fix works for me:
http://github.com/JonnyJD/rubyripper/commit/a1ab5c56816bc7c32bcdef61035b74277323
8298

Original comment by goo...@JonnyJD.net on 4 Jun 2010 at 10:23

GoogleCodeExporter commented 9 years ago 
Thanks for finding the problem. I've solved it a little differently (yeah, 
again, I 
know):
http://github.com/rubyripperdev/rubyripper/commit/67d6650ed3bd35273f482172cfc0be
bc3022
f3c2

Original comment by boukewou...@gmail.com on 5 Jun 2010 at 9:23