louismullie / treat

Natural language processing framework for Ruby.
Other
1.36k stars 128 forks source link

invalid multibyte char (US-ASCII) #39

Closed seyhunak closed 11 years ago

seyhunak commented 11 years ago

I'm getting this error below, after requiring treat in my irb console

1.9.3-p0 :001 > require 'treat'
SyntaxError: (eval):60: invalid multibyte char (US-ASCII)
(eval):60: invalid multibyte char (US-ASCII)
(eval):60: syntax error, unexpected $end, expecting ']'
      "für",
          ^

What is the problem here,

Environment: ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-darwin11.0.0] treat (2.0.3)

Thanks

louismullie commented 11 years ago

I'm not able to reproduce with 1.9.3-p194.

louis-mullies-macbook-pro:treat Louis$ irb
1.9.3-p194 :001 > require "treat"
 => true 
1.9.3-p194 :002 > Treat::VERSION
 => "2.0.3" 
1.9.3-p194 :003 > Treat.languages.german.stop_words
 => ["alle", "allem", "alles", "andere", "anderem", "anderen", "anderer", "anderes", "auf", "bei", "beim", "bist", "dadurch", "dein", "deine", "deiner", "deines", "deins", "dem", "denen", "der", "deren", "des", "deshalb", "dessen", "diese", "diesem", "diesen", "dieser", "dieses", "ein", "eine", "einem", "einen", "einer", "eines", "euer", "euere", "eueren", "eueres", "für", "haben", "habt", "hatte", "hatten", "hattest", "hattet", "hierzu", "hinter", "ich", "ihr", "ihre", "ihren", "ihrer", "ihres", "indem", "ist", "jede", "jedem", "jeden", "jeder", "jedes", "kann", "kannst", "können", "könnt", "konnte", "konnten", "konntest", "konntet", "mehr", "mein", "meine", "meiner", "meines", "meins", "nach", "neben", "nicht", "nichts", "seid", "sein", "seine", "seiner", "seines", "seins", "sie", "sind", "über", "und", "uns", "unser", "unsere", "unter", "vor", "warst", "weil", "wenn", "werde", "werden", "werdet", "willst", "wir", "wird", "wirst", "wollen", "wollt", "wollte", "wollten", "wolltest", "wolltet", "zum", "zur"]
louismullie commented 11 years ago

Can't reproduce on 1.9.2p320 or 1.9.3p327 either. Could this be a patch-level-specific Ruby bug?

seyhunak commented 11 years ago

@louismullie wont work on "ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin12.2.0]" What is the workaround?

louismullie commented 11 years ago

For a workaround, just delete everything in /treat/config/data/languages except english.rb.

However, I would love to find out why this is not working for you. Can you go through the following?

web028013:~ Louis$ rvm use 1.9.3-p194
Using /Users/Louis/.rvm/gems/ruby-1.9.3-p194
web028013:~ Louis$ ruby -v
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin11.3.0]
web028013:~ Louis$ irb -v
irb 0.9.6(09/06/30)
web028013:~ Louis$ gem install treat
Successfully installed treat-2.0.3
1 gem installed
Installing ri documentation for treat-2.0.3...
Installing RDoc documentation for treat-2.0.3...
web028013:~ Louis$ irb
1.9.3p194 :001 > require 'treat'
 => true 
1.9.3p194 :002 > puts Treat::VERSION
2.0.3
 => nil 
seyhunak commented 11 years ago

@louismullie deleting everything except "english.rb" is working. Thanks.

louismullie commented 11 years ago

Cool.