search

louismullie / treat

Natural language processing framework for Ruby.
Other
1.37k stars 128 forks source link

Treat::Exception: No TF_IDF is/are available for the English Language? #25

Closed bckwon83 closed 11 years ago

bckwon83 commented 11 years ago

Hello,

First of all, thank you so much for creating this awesome gem. I have one issue while using this gem. I have trouble getting keywords out of collection. I just used the sample code you provided in the tutorial:

c = Collection('economist') c.do(:chunk, :segment, :tokenize, :keywords)

I have installed tf_idf gem manually and it is in place. However, I still get this weird error:

Treat::Exception: No tf_idf is/are available for the English language. from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/delegatable.rb:112:in find_worker_for_language' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/delegatable.rb:85:infind_worker' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/delegatable.rb:57:in call_worker' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/delegatable.rb:42:inblock (2 levels) in add_workers' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/workers/extractors/keywords/tf_idf.rb:20:in block in keywords' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/magical.rb:45:inblock in magic' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:22:in block (2 levels) in each_entity' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:22:inblock (2 levels) in each_entity' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:22:in block (2 levels) in each_entity' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:18:ineach_entity' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:22:in block in each_entity' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:21:ineach' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:21:in each_entity' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:22:inblock in each_entity' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:21:in each' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/iterable.rb:21:ineach_entity' ... 5 levels... from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/workers/extractors/keywords/tf_idf.rb:19:in keywords' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/delegatable.rb:68:incall_worker' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/delegatable.rb:42:in block (2 levels) in add_workers' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/doable.rb:39:indo_task' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/doable.rb:47:in block in do_task' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/doable.rb:46:ineach' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/doable.rb:46:in do_task' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/doable.rb:22:inblock in do' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/doable.rb:6:in each' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/treat-1.2.0/lib/treat/entities/abilities/doable.rb:6:indo' from (irb):2 from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/railties-3.2.6/lib/rails/commands/console.rb:47:in start' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/railties-3.2.6/lib/rails/commands/console.rb:8:instart' from /Users/kwonb/.rvm/gems/ruby-1.9.3-p0/gems/railties-3.2.6/lib/rails/commands.rb:41:in <top (required)>' from script/rails:6:inrequire' from script/rails

Sorry, I might be throwing a too basic question, but this problem cannot be solved by myself.. is there some additional steps I need to do?

driki commented 11 years ago

Hi,

I'm also getting the same message.

1.9.3-p125 :332 > doc = Document('/tmp/document.txt') => Document (70334188576300) --- "NORWICH DEVELOPMENT REVIEW [...] Dechert, Clerk" --- {:file=>"/tmp/document.txt", :format=>"txt"} --- []

1.9.3-p125 :333 > doc.do :chunk, :segment, :tokenize, :keywords Treat::Exception: No tf_idf is/are available for the English language.

darkphantum commented 11 years ago

It looks like the files downloaded using 'gem install treat' are stale

replace the following files in your gem directory from the github repo.

lib\treat\config\data\languages\agnostic.rb lib\treat\config\data\languages\english.rb lib\treat\workers\extractors\tf_idf\native.rb

louismullie commented 11 years ago

Yup, that should be the main problem. Although I can't guarantee everything to be compatible, since there was a major update to the configuration files since the last gem release. Thinks are looking good for me to publish the next version of the gem around this week-end, so stay tuned for that.

louismullie commented 11 years ago

Fixed in the master branch.