search

ariyatan / Tibetan

Python 2.7 scripts for processing Tibetan language, and necessary data. I've developped these as help functions for current bigger project, but I want to make it public in case someone else may use or test it. It has worked just fine for my needs, my data and my system, but may have errors in other circomstances. If you use my modules and find any errors or lack of important features - feel free to write me.
GNU General Public License v2.0
7 stars 0 forks source link

examples for tokenizer #1

Open tenzinngodup opened 8 years ago

tenzinngodup commented 8 years ago

do you have any examples for tokenizer of Tibetan Scripts?

ariyatan commented 8 years ago

do you mean sample of text in Tibetan script, parsed by this tokenizer? I mainly use it as a part of pos-tagger, but i have a sample of tokenized text without annotation made as a test, and can send it via email.

tenzinngodup commented 8 years ago

Hello there,

Sure, that would be great!

Thanks, Ngodup

On Thu, May 12, 2016 at 12:18 AM, ariyatan notifications@github.com wrote:

do you mean sample of text in Tibetan script, parsed by this tokenizer? I mainly use it as a part of pos-tagger, but i have a sample of tokenized text without annotation made as a test, and can send it via email.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/ariyatan/Tibetan/issues/1#issuecomment-218678861