search

aschaeffer / dkpro-core-asl

Automatically exported from code.google.com/p/dkpro-core-asl
0 stars 0 forks source link

[TreeTagger] Problem processing very long tokens #30

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
When there is a very very long token in the text, the analysis engine fails.

Original issue reported on code.google.com by richard.eckart on 2 Oct 2011 at 3:14

GoogleCodeExporter commented 9 years ago 
- Upgrade to TT4J 1.0.15 (which handles very long tokens)
- Added log-guard in TreeTaggerPosLemma to avoid unnecessary construction of 
TT4J status string
- Added a test case for very long tokens
---
rev 109

Original comment by richard.eckart on 2 Oct 2011 at 3:15