search

kraison / langutils

NLP for Common Lisp
http://common-lisp.net/project/langutils/
Other
20 stars 3 forks source link

Period not correctly tokenized? #1

Open huangjs opened 13 years ago

huangjs commented 13 years ago

Here's an example:

LANGUTILS> (tokens-for-ids (vector-document-words (vector-tag "Hello world. I'm here."))) ("Hello" "world." "I" "'" "m" "here.")

I think it should be:

("Hello" "world" "." "I" "'" "m" "here" ".")

kraison commented 13 years ago

Would you mind using the mailing list to discuss this? You can join it here:

http://lists.common-lisp.net/mailman/listinfo/langutils-devel

Cheers, Kevin

huangjs commented 13 years ago

On Sat, Aug 13, 2011 at 11:16 AM, kraison reply@reply.github.com wrote:

Would you mind using the mailing list to discuss this?  You can join it here:

http://lists.common-lisp.net/mailman/listinfo/langutils-devel

Sure

黄　澗石 (Jianshi Huang) http://huangjs.net/