tarsqi / ttk

Tarsqi Toolkit
Apache License 2.0
25 stars 10 forks source link

Clarify preprocessor modules #37

Closed marcverhagen closed 7 years ago

marcverhagen commented 7 years ago

It may be good to point out that the results may be weird when you use both TOKENIZER and PREPROCESSOR in your pipeline. Should actually check whether this breaks or whether downstream components deal with this gracefully.