Change: -multidoc [SEP] option for all programs now regards lines "beginning
with" [SEP] as comment lines and print them without any modification,
whereas it previously dealt with the lines "exactly same" to [SEP].
In addition, multiple comment lines can be used from this version.
Change: nersuite_tokenizer computes token offsets at document level, whereas
it used sentence level token offsets. When it encounters a document
separator, which is given by -multidoc [SEP] option, the tokenizer
resets the offset to 0.
New function: correct_mp program which removes named entities having mismatched
parenthesis is added. Documentation will be added to the homepage
later.
For full history, please refer to README file
Version 1.2.1 (2012.07.02)