Closed CpuID closed 10 years ago
I've never seen this error before. Is there any chance you could provide me with that file? And ideally the exact invocation of the FileMagic method you used.
No problem. See below for how I have executed it:
nsullivan:rbxtest nsullivan$ chmod +x test.rb
nsullivan:rbxtest nsullivan$ ./test.rb
An exception occurred running ./test.rb:
line 66: regex error 10, (invalid repetition count(s)) (FileMagic::FileMagicError)
Backtrace:
Object#__script__ at test.rb:7
Rubinius::CodeLoader#load_script at kernel/delta/code_loader.rb:66
Rubinius::CodeLoader.load_script at kernel/delta/code_loader.rb:152
Rubinius::Loader#script at kernel/loader.rb:649
Rubinius::Loader#main at kernel/loader.rb:831
nsullivan:rbxtest nsullivan$ rbenv versions
system
1.8.7-p352
1.8.7-p374
1.9.3-p484
2.0.0-p353
2.1.1
rbx-2.2.5
* rbx-2.2.6 (set by /Users/nsullivan/rbxtest/.ruby-version)
nsullivan:rbxtest nsullivan$ rm .ruby-version
nsullivan:rbxtest nsullivan$ rbenv versions
system
1.8.7-p352
1.8.7-p374
1.9.3-p484
* 2.0.0-p353 (set by /usr/local/var/rbenv/version)
2.1.1
rbx-2.2.5
rbx-2.2.6
nsullivan:rbxtest nsullivan$ ./test.rb
./test.rb:7:in `file': line 66: regex error 10, (invalid repetition count(s)) (FileMagic::FileMagicError)
from ./test.rb:7:in `<main>'
nsullivan:rbxtest nsullivan$ cat test.rb
#!/usr/bin/env ruby
require 'ruby-filemagic'
require 'pp'
fm = FileMagic.new(FileMagic::MAGIC_MIME)
file_mime_type = fm.file('/Users/nsullivan/.rbx/09/097e79c388cab23f8981958bea209efff565ae1b')
pp file_mime_type
I'll email you a URL to the file itself, I don't believe it has anything sensitive but just to be safe I won't put it in this public ticket.
URL to pull the file emailed to your address on your github account.
Great, thanks. Unfortunately, I could not reproduce the error with ruby 2.0.0p377, nor ruby 2.1.2p77, nor rubinius 2.2.6 (2.1.0) on x86_64-linux-gnu. Can you try different ruby versions? (2.1.1, 1.9.3-p484) What libmagic version do you have? (5.14 here) What happens if you remove the offending line?
(And just for the record: The tests pass on ruby 2.0.0p353 and rubinius 2.2.6 with libmagic 5.09.)
Interesting. I am running Ruby on Mac OSX here, using HomeBrew for libmagic, looks like 5.18
nsullivan:~ nsullivan$ brew info libmagic
libmagic: stable 5.18 (bottled)
http://www.darwinsys.com/file/
/usr/local/Cellar/libmagic/5.18 (14 files, 2.9M) *
Poured from bottle
From: https://github.com/Homebrew/homebrew/commits/master/Library/Formula/libmagic.rb
I have just given it a try on 2.1.1:
nsullivan:rbxtest nsullivan$ rbenv local 2.1.1
nsullivan:rbxtest nsullivan$ bundle install
Fetching gem metadata from https://rubygems.org/.........
Fetching additional metadata from https://rubygems.org/..
Installing builder 3.2.2
Installing excon 0.31.0
Installing formatador 0.2.4
Installing mime-types 2.1
Installing multi_json 1.10.0
Installing net-ssh 2.9.0
Installing net-scp 1.2.1
Installing mini_portile 0.5.3
Installing nokogiri 1.6.1
Installing fog 1.20.0
Installing ruby-filemagic 0.5.2
Installing unf_ext 0.0.6
Installing unf 0.1.3
Using bundler 1.6.1
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
Post-install message from ruby-filemagic:
ruby-filemagic-0.5.2 [2014-04-24]:
* Use MAGIC_VERSION if available.
* Made tests more robust.
nsullivan:rbxtest nsullivan$ ./test.rb
./test.rb:7:in `file': line 66: regex error 10, (invalid repetition count(s)) (FileMagic::FileMagicError)
from ./test.rb:7:in `<main>'
nsullivan:rbxtest nsullivan$ rbenv versions
system
1.8.7-p352
1.8.7-p374
1.9.3-p484
2.0.0-p353
* 2.1.1 (set by /Users/nsullivan/rbxtest/.ruby-version)
rbx-2.2.5
rbx-2.2.6
I couldn't get 1.9.3-p484 tested due to some gem failures, see below:
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
/usr/local/var/rbenv/versions/1.9.3-p484/bin/ruby extconf.rb
checking for main() in -lstdc++... yes
checking for ruby/encoding.h... yes
creating Makefile
make
compiling unf.cc
cc1plus: error: -Werror=shorten-64-to-32: no option -Wshorten-64-to-32
make: *** [unf.o] Error 1
Gem files will remain installed in /usr/local/var/rbenv/versions/1.9.3-p484/lib/ruby/gems/1.9.1/gems/unf_ext-0.0.6 for inspection.
Results logged to /usr/local/var/rbenv/versions/1.9.3-p484/lib/ruby/gems/1.9.1/gems/unf_ext-0.0.6/ext/unf_ext/gem_make.out
An error occurred while installing unf_ext (0.0.6), and Bundler cannot continue.
Make sure that `gem install unf_ext -v '0.0.6'` succeeds before bundling.
Looks like it was gcc 4.2.1 that Xcode has installed I believe being used, but I could be wrong here... Obviously let's keep this bug to working out why the regex fails rather than why my build of the gem doesnt work for now :)
Well, that would suggest that it might have to do with changes in libmagic. But again I wasn't able to reproduce with 5.18 either.
As for your gem install failure: That's unf_ext
which is unrelated to ruby-filemagic
. I don't think that we have to look at other Ruby versions, though. It seems safe to say that it's platform related somehow, so we can concentrate on 2.1.
Seeing as I don't have access to a Mac, could you try the following and report with your results:
In your shell:
file -v
In irb -rfilemagic
:
FileMagic::MAGIC_VERSION
FileMagic.path
from 0.5.2.pre1FileMagic.mime.file('test.rb')
FileMagic.mime.buffer(File.read('/Users/nsullivan/.rbx/09/097e79c388cab23f8981958bea209efff565ae1b', 209))
FileMagic.mime.buffer(File.read('/Users/nsullivan/.rbx/09/097e79c388cab23f8981958bea209efff565ae1b', 218))
Dir['/Users/nsullivan/.rbx/*/*'].find { |f| FileMagic.mime.file(f) rescue nil }
and send me that file if anyI have a feeling that this is not an issue with ruby-filemagic, but rather with your magic file(s) -- the error message seems to indicate as much. Alas, then why would the file
command succeed? I hope we can get to the bottom of it with some back-and-forth.
Awesome, thanks. See below:
nsullivan:~ nsullivan$ file -v
file-5.04
magic file from /usr/share/file/magic
And from irb -rfilemagic:
nsullivan:rbxtest nsullivan$ irb -rfilemagic
irb(main):001:0> FileMagic::MAGIC_VERSION
=> "5.18"
irb(main):003:0> FileMagic.mime.file('test.rb')
=> "text/x-ruby; charset=us-ascii"
irb(main):004:0> FileMagic.mime.buffer(File.read('/Users/nsullivan/.rbx/09/097e79c388cab23f8981958bea209efff565ae1b', 209))
=> "text/plain; charset=us-ascii"
irb(main):005:0> FileMagic.mime.buffer(File.read('/Users/nsullivan/.rbx/09/097e79c388cab23f8981958bea209efff565ae1b', 218))
=> "text/plain; charset=us-ascii"
irb(main):006:0> Dir['/Users/nsullivan/.rbx/*/*'].find { |f| FileMagic.mime.file(f) rescue nil }
=> "/Users/nsullivan/.rbx/00/000144baa96257cbd535c3b229cdfb4ffcecf3fb"
irb(main):007:0> exit
When you said FileMagic.path from 0.5.2.pre1, as in install that gem and try running it? Running 0.5.2 already, is pre1 older?
Emailing you /Users/nsullivan/.rbx/00/000144baa96257cbd535c3b229cdfb4ffcecf3fb in the meantime.
Um, now that's less helpful than I had hoped for... Let's step back for a moment:
file -i <offending-file>
works.FileMagic.mime.file/.buffer
works in some cases (even of the same kind), but not with <offending-file>
.buffer
above to fail. My guess was that it refers to the magic file(s). Can you send me the file in /usr/share/file/magic
and maybe also the ones in FileMagic.path
? I may be on the wrong track, though.I seem to recall that one can have competing/conflicting versions of libraries on OS X (e.g., system version vs. homebrew version). Is it possible that file
picks up a different one than FileMagic
?
Other than that, I'm afraid I'm out of ideas. Let me think about it for a while. But feel free to poke around some more:
At which line does <offending-file>
fail?
fm, b = FileMagic.new(FileMagic::MAGIC_MIME), ''
File.foreach('/Users/nsullivan/.rbx/09/097e79c388cab23f8981958bea209efff565ae1b') { |l| fm.buffer(b << l) rescue break }
p $.
Which of the .rbx
files work and which don't?
ok, nok = Dir['/Users/nsullivan/.rbx/*/*'].partition { |f| fm.file(f) rescue nil }
p [ok.size, nok.size]
etc.
When you said FileMagic.path from 0.5.2.pre1, as in install that gem and try running it? Running 0.5.2 already, is pre1 older?
Yeah, that's what I meant. Forgot to increment the version number ;) I just pushed 0.5.3.pre1 instead.
By now I'm pretty confident that it's indeed an issue with one of your magic files, not with these bindings. line 66
refers to the "current magic line number", we just have to find out which file it is.
The fact that file
behaves differently than FileMagic
may still indicate a problem, but that should hardly matter. Just to be sure: All the commands have been executed on the same machine by the same user, right?
One more thing to look at:
strace -e open 2>&1 ruby -rfilemagic -e 'FileMagic.mime.file("/Users/nsullivan/.rbx/09/097e79c388cab23f8981958bea209efff565ae1b")' | grep [./]magic
and
strace -e open 2>&1 file --mime-type /Users/nsullivan/.rbx/09/097e79c388cab23f8981958bea209efff565ae1b | grep [./]magic
(The equivalent of strace -e open
on OS X should be dtruss -t open
.)
Getting this on a specific file, which file --mime-type returns "text/plain" for currently.
This error occurs under MRI 2.0.0-p353 and Rubinius 2.2.6.
Any ideas?