search

Kipok / NeMo-Skills

A pipeline to improve skills of large language models
https://kipok.github.io/NeMo-Skills/
Apache License 2.0
185 stars 41 forks source link

Multiple fixes #81

Closed Kipok closed 2 months ago

Kipok commented 3 months ago
  1. Remove "context" parameter as it's not really used anymore and is just confusing for people.
  2. Update trtllm arguments in the docs
  3. Add llama3 chat format support to sft and data preparation. This is a bit hacky at the moment and will be fixed when this is properly supported in nemo
  4. Remove system message from llama3 sft config
  5. Fix gpu tests and upgrade finetuning test to llama3