korean pre-processing (pos tagger)

[kozistr]

There're lots of Korean-morph-analyzers like Twitter, Mecab, Hannanum, etc...

Try all of them! But, the dataset in the wild isn't verified, not clean and of course, there're lots of coned-word. So, a proper analyzer is needed.

• [x] Hannanum (not bad but memory + speed stuff...,)
• [x] Mecab (not bad)
• [x] Twitter (i think it's not good)
• [o] soynlp (needs to try out)

[kozistr]

I think using soynlp (L-Tokenizer) should be used after dealing with spacing problem.

Later, I'll try!