Updates for Llama 3.1 compatibility, import error workarounds & more

[AgenP]

Train.py

• Axolotl base image SHA updated, and pinned, to a Llama 3.1 compatible version
• Allow background commits flag removed since deprecated
• Regression test: No errors when running this file using llama-3.yml

Inference.py

• Updated vLLM & PyTorch to required versions for Llama 3.1 support due to a RoPE scaling configuration error (see https://github.com/vllm-project/vllm/pull/6693)
• Regression test: No errors when using the same Llama 3 axolotl run file generated from the train.py regression test

Common.py

Have been unable to get past a ModuleNotFoundError importing common.py

So I commented out the import and directly pasted the relevant code into both train.py & inference.py

Apologies if I’ve missed something obvious for the import as this doesn’t seem like the most elegant fix

But submitting it, cause as recently been said, fear is the mind-killer

(On a real note, more than happy to iterate if I’ve missed something here)

Requirements.txt (local usage)

• Modal
• PyYaml (used in inference.py)
• Pandas (Commented out, but used for CI in check_loss.py)

Llama 3.1 yaml file added

CI files

• check_loss.py ran fine
• check_inference unable to get past ModuleNotFoundError (However inference.py worked w/ Llama 3.1, Llama 3 & Open-Llama-7b-v2)