1. Setup german LLM Fork (Baseline)

[Kuckuck44]

• Pick a model which is already able to speak German in reasonable quality
• Maybe a LLAMA-Fork?
• Get model ready to output text from starting-prompt (no chat-finetuning)
• Maybe also fine-tune German model by ourselves

[Kuckuck44]

• Picked German LLAMA-Fork by "TheBloke" (known contributor on Hugging Face) https://huggingface.co/TheBloke/Llama-2-13B-German-Assistant-v4-GGUF/tree/main

• Model, which was tested: llama-2-13b-german-assistant-v4.Q2_K.gguf

• Used desktop-version of GPT4All (https://gpt4all.io/index.html) to generate Text

• GPT4All is a library / tool for easy model-interaction → Allows use of different models with low effort

• Makes use / interchangeability of German baseline easy

How to import own models:

1. Download and configure GPT4All with the help of the installer
2. Open Settings → Navigate to "Application"
3. Copy the path behind "Download Path" or customize it

4. Copy the downloaded model into this path to use it in the desktop-client

Results:

• German language of the tested model is not very sophisticated (arising question: how "good" should our baseline be?)
• Method suitable for our use case in my opinion

[Kuckuck44]

For further testing I will try llama-2-13b-german-assistant-v4.Q4_K_M.gguf, as it is flagged as "medium, balanced quality - recommended".

By the way - "TheBloke" only provided the "inferencable" model formats. The underlaying model is fine-tuned by Florian Zimmermeister.

[Kuckuck44]

To generate the first test-data, i will use the Prompt "Bitte generiere einen langen Aufsatz, basierend auf dem folgenden Anfang: "Hallo, mein Name ist"." The Max-Token paramter for the generated response has ben manually modified to 131.072, instead of the default 4096.

I will create a .txt file with at least 5.000 Words from different responses to that input...anything else will last forever :)

[Kuckuck44]

Just to clarify: the pretty silly modification of the "Max-Token" parameter was just in regard to some testing of GPT4All, because the used model behaved strangely at first.