5. Evaluate captured data

[Kuckuck44]

• Now compare the different approaches (German LLM, English with plain book, English with weeve, English with own diglot-weave)
• Measure Vocabulary size and frequency in generated texts
• Measure spelling and grammar mistakes

Generate more output, if necessary → Texts compared should have approximately the same length

[Kuckuck44]

I prepared a spellchecking notebook here:

https://colab.research.google.com/drive/1sO7BPltMPIN_0h_WOUOllVSil5KCm3RW?usp=sharing

[Powerkrieger]

extracted notebook to spellchecking script. Evaluation is ongoing

[Powerkrieger]

inference tasks:

• prompt : 'Bitte generiere einen langen Aufsatz, basierend auf dem folgenden Anfang: "Hallo, mein Name ist".'

  • [ ] lvl2
  • [ ] lvl3
  • [ ] lvl4
  • [ ] textbook
  • [ ] llama-base
  • [ ] (optional) german baseline

• prompt :"Write a story about a sailor. "

  • [ ] lvl2
  • [ ] lvl3
  • [ ] lvl4
  • [ ] textbook
  • [ ] llama-base
  • [ ] (optional) german baseline

[Powerkrieger]

• [ ] evaluation of all of the above