We should rip out chunk tokenizer - it uses the old Chunker APIs. With the new
Chunking APIs this extra boilerplate is unnecessary. (The chunk tokenizer was
deprecated by Issue 302, this issue is to remind us to get rid of it entirely.)
Original issue reported on code.google.com by steven.b...@gmail.com on 21 Apr 2012 at 6:55
Original issue reported on code.google.com by
steven.b...@gmail.com
on 21 Apr 2012 at 6:55