Kimtaro / ve

A linguistic framework that's easy to use.
MIT License
215 stars 25 forks source link

Help to resolve installation problem #31

Open psychoslave opened 6 years ago

psychoslave commented 6 years ago

Hello,

I wanted to test ve, specificaly to parse French sentences. I installed FreeLing-4.0 from source on Fedora 27, as following

sudo dnf install boost-devel
wget https://github.com/TALP-UPC/FreeLing/releases/download/4.0/FreeLing-4.0.tar.gz
tar xvzf FreeLing-4.0.tar.gz
cd FreeLing-4.0
autoreconf --install
./configure
make
sudo make install

Then I created a new directory with the following Gemfile

source 'https://rubygems.org'

gem "ve"

Then running bundle install && pry, here is a copy/paste of a basic session:

[1] pry(main)> require 've'
/home/psychoslave/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/ve-0.0.3/lib/providers/japanese_transliterators.rb:100: warning: key "gwi" is duplicated and overwritten on line 101
/home/psychoslave/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/ve-0.0.3/lib/providers/japanese_transliterators.rb:100: warning: key "gwu" is duplicated and overwritten on line 101
/home/psychoslave/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/ve-0.0.3/lib/providers/japanese_transliterators.rb:100: warning: key "gwe" is duplicated and overwritten on line 101
/home/psychoslave/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/ve-0.0.3/lib/providers/japanese_transliterators.rb:100: warning: key "gwo" is duplicated and overwritten on line 101
=> true
[2] pry(main)> text = %q{Socrate (en grec ancien Σωκράτης / Sōkrátēs) est un philosophe grec du Ve siècle av. J.-C. (né vers -470/469, mort en -399). Il est connu comme l’un des créateurs de la philosophie morale. Socrate n’a laissé aucun écrit, mais sa pensée et sa réputation se sont transmises par des témoignages indirects. Ses disciples Platon et Xénophon ont notablement œuvré à maintenir l'image de leur maître, qui est mis en scène dans leurs œuvres respectives. Les philosophes Démétrios de Phalère, et Maxime de Tyr dans sa Neuvième Dissertation1 ont écrit que Socrate est mort à l’âge de 70 ans.}
=> "Socrate (en grec ancien Σωκράτης / Sōkrátēs) est un philosophe grec du Ve siècle av. J.-C. (né vers -470/469, mort en -399). Il est connu comme l’un des créateurs de la philosophie morale. Socrate n’a laissé aucun écrit, mais sa pensée et sa réputation se sont transmises par des témoignages indirects. Ses disciples Platon et Xénophon ont notablement œuvré à maintenir l'image de leur maître, qui estémis en scène dans leurs œuvres respectives. Les philosophes Démétrios de Phalère, et Maxime de Tyr dans sa Neuvième Dissertation1 ont 
 crit que Socrate est mort à l’âge de 70 ans."
[3] pry(main)> words = Ve.in(:fr).words(text)
NoMethodError: undefined method `[]' for nil:NilClass
from /home/psychoslave/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/ve-0.0.3/lib/ve.rb:20:in `provider_for'
[4] pry(main)> words = Ve.in(:en).words('I like melons.')
=> []

Given this, what should be checked? It seems the French parser doesn't work at all, while the English one just provides completely irrelevant results with an empty array.

Kimtaro commented 6 years ago

Hi, sorry for the late reply. I was out traveling without my computer.

Unfortunately Ve only supports Freeling in English at this point. You would have to create a new provider file, similar to https://github.com/Kimtaro/ve/blob/master/lib/providers/freeling_en.rb, and modify it to work with Freeling's French parser output.

I don't have Freeling installed at the moment so I can't check, but if the French parser has the exact same type of output as the English parser, then you could trick the freeling_en.rb file to use French instead by making these changes:

On https://github.com/Kimtaro/ve/blob/master/lib/providers/freeling_en.rb#L24 change en.cfg to fr.cfg On https://github.com/Kimtaro/ve/blob/master/lib/providers/freeling_en.rb#L231 change :en to :fr

I hope this helps.

psychoslave commented 6 years ago

Thank you very much @Kimtaro for your reply. Right now I have to focus on other priorities, but your feedback will certainly help me when I can allocate again some time to the project for which I was making this test.