search

zencephalon / Tactful_Tokenizer

Accurate Bayesian sentence tokenizer in Ruby.
80 stars 13 forks source link

Error loading with 1.9.3 #9

Closed revelutions closed 9 years ago

revelutions commented 12 years ago

I get an error when creating a new model. I assume this is version depedent?

irb(main):006:0> m = TactfulTokenizer::Model.new "C:/Ruby193/lib/ruby/gems/1.9.1/gems/tactful_tokenizer-0.0.2/lib/models/features .mar" ArgumentError: dump format error(0x67) from C:/Ruby193/lib/ruby/gems/1.9.1/gems/tactful_tokenizer-0.0.2/lib/tac tful_tokenizer.rb:56:in load' from C:/Ruby193/lib/ruby/gems/1.9.1/gems/tactful_tokenizer-0.0.2/lib/tac tful_tokenizer.rb:56:inblock (2 levels) in initialize' from C:/Ruby193/lib/ruby/gems/1.9.1/gems/tactful_tokenizer-0.0.2/lib/tac tful_tokenizer.rb:55:in open' from C:/Ruby193/lib/ruby/gems/1.9.1/gems/tactful_tokenizer-0.0.2/lib/tac tful_tokenizer.rb:55:inblock in initialize' from C:/Ruby193/lib/ruby/gems/1.9.1/gems/tactful_tokenizer-0.0.2/lib/tac tful_tokenizer.rb:53:in map' from C:/Ruby193/lib/ruby/gems/1.9.1/gems/tactful_tokenizer-0.0.2/lib/tac tful_tokenizer.rb:53:ininitialize' from (irb):6:in new' from (irb):6 from C:/Ruby193/bin/irb:12:in

' irb(main):007:0>

zencephalon commented 12 years ago

Hm, looks like there is an encoding incompatibility between Linux and Windows.

I may be able to resolve this issue using info from here: http://stackoverflow.com/questions/7649175/why-does-dumping-and-loading-a-hash-using-marshal-in-ruby-throw-a-formaterror