task: Ichigo Training

[hahuyhoang411]

Goal

Experiment on Ichigo model on multilingual.

Tasklist

• [x] https://github.com/janhq/ichigo/issues/125
• [x] Experiment with hyper-parameters
• [ ] Draft research report
• [ ] Check for the degradation
• [ ] Check the model can do multi-lingual conversation

[hahuyhoang411]

Lora for pretraining ref: https://unsloth.ai/blog/contpretraining

[tuanlda78202]

Paired with @bachvudinh to understand the fully trained original Ichigo pipeline.

[dan-homebrew]

@hahuyhoang411 - Can we combine this into #116