Handling Named Entities

[ybracke]

Learn about and experiment with methods for proper handling of named entities.

Named entities (NE) may be distorted during normalization, e.g. das Samoliſche Kraut -> das Salomischen Kraut, Cejonius -> Ce Kostenius.

To improve this, we have to learn more about existing methods to handle NE in neural machine translations. Check out these papers and look for more:

• [ ] Koehn et al. 2017 (general)
• [ ] Modrzejewski et al. 2020
  • See also this blogpost on the paper.
• [ ] Mota et al. 2022 (handling of NE as a separate module in MT pipeline)

[ybracke]

Software:

• [ ] flair is state-of-the art for modern German NER
• [ ] Overview of tools (some historic)
• [ ] Model for "historical" German NER (around 1900)
• [ ] historic-ner (dbmdz)
• [ ] sequence_tagging (apparently for historic text as well)

[ybracke]

Conceptually: How can a pre-trained encoder (decoder) deal with previously recognized NE? (This question extends to previously recognized foreign material, etc.