Language Detection

[carschno]

Add a language detection mechanism. If the language of a document is not as expected (e.g. Dutch), do not run the machine-learning based, language-specific quality classification.

• [x] Add language classification, e.g. with FastText and/or CLD3, LangId
• [x] Add output column 'Language'
• [x] Skip classification if language is not Dutch (parametrize for other languages)
• [x] Add possible reason value 'LANGUAGE'

[kintopp]

Perhaps of interest: https://www.cenl.org/wp-content/uploads/2023/04/CENL_AI_Yves_Maurer.pdf