taranjeet / hindi-tokenizer

This is a package in Python which implements a tokenizer, stemmer for Hindi language
90 stars 75 forks source link

Error at hindi-tokenizer #1

Open lubnajava opened 6 years ago

lubnajava commented 6 years ago

An excellent initiative in the area of WSD (Word Sense Disambiguation). Excited to run the code , but unable to do so , giving some errors ,

I have "devnagri script file" as a dataset for WSD task...I want to read the file through your Hindi-Tokenizer ...Error at line 36 , which is shown in the screenshot..

from HindiTokenizer import Tokenizer t=Tokenizer() t.read_from_file('ContextSenses001.txt')

t=Tokenizer("यह वाक्य हिन्दी में है।")

The screenshot is attached here with.. error

Kindly resolve the same at the earliest... Thanks