Multi-word named entities as a single token

[GoogleCodeExporter]

There should be some way to turn multi-word named entities into a single token. 
I think there are two options:

1) change any NER component so that it can optionally merge all tokens covered 
by a named entity into a single token

2) implement a new component that takes a type (e.g. NamedEntity) and merges 
all tokens covered by that type into a single token.

I think 2) would be the better solution.

Original issue reported on code.google.com by richard.eckart on 1 Jul 2012 at 6:16

[GoogleCodeExporter]

I like option 2)

Original comment by torsten....@gmail.com on 1 Jul 2012 at 6:59

[GoogleCodeExporter]

Started with 2)... what I thought would be a most simple component turned out 
to be quite a bit more complicated - I'm also adding handling of POS tags and 
lemmata.

Original comment by richard.eckart on 3 Jul 2012 at 7:51

• Changed title: [tokit] Multi-word named entities as a single token
• Changed state: Started
• Added labels: Milestone-1.4.0

[GoogleCodeExporter]

Original comment by richard.eckart on 13 Oct 2012 at 6:31

• Added labels: DKPro-ASL

[GoogleCodeExporter]

Original comment by richard.eckart on 13 Oct 2012 at 6:33

• Added labels: Milestone-1.5.0
• Removed labels: Milestone-1.4.0

[GoogleCodeExporter]

Original comment by richard.eckart on 16 Feb 2013 at 11:02

• Changed title: Multi-word named entities as a single token
• Added labels: Module-tokit

[GoogleCodeExporter]

I think this can be called done for the moment. A basic implementation with 
some tests is there. Better open new issues for any problems/extensions of this 
component.

Original comment by richard.eckart on 10 Apr 2013 at 4:19

• Changed state: Fixed