Synthetic training data

[ybracke]

Possible strategies:

• Create near duplicates from orig layer of a normalized dataset, e.g. with augtxt
• "Translate" modern text to historical text. Options:
  • Using CAB's expand mechanism.
  • Can we use a Norma model?
  • Can we use a seq2seq transformer in the reverse order? -> probably not, because encoder and decoder are not build symmetrically -> but we can train a model in reverse order (norm as source, orig as target)
  • Can we use/train a csmtiser statistical normalizer in the reverse order?

Material:

• Blog post on synthetic training data for NMT

[ybracke]

Norma might be the easiest approach, see ~/code/norma/HOWTO.md

• potential drawback: translation is deterministic + can't do k-best (unlike a transformer)
• it might be reasonable to not use the lexicon option (but I'm not sure - just check both)

What we would need:

• Data to train Norma

  • format: token-aligned tsv, reversed (norm\torig)
  • available data: dta eval, ridges bollmann, perhaps GerMan-C

• historic lexicon as "target" lexicon (create from dta, does something exist already?)

• modern text to convert into historic variant

  • available data: leipzig zeitung corpora; books, newspapers, etc from the DWDS
  • convert to correct format, e.g. by using the WASTE tokenizer or any other tokenizer that gives you sentence boundaries and character offsets

• serialization script to convert single token text plus byte offsets back to sentences