search

Lightning-AI / lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
Apache License 2.0
5.97k stars 518 forks source link

Fix: Variable name #476

Open dhdbsrlw opened 9 months ago

dhdbsrlw commented 9 months ago

Hello.

I think variable's name should be changed in 'lit_llama/utils.py.'

When I implement your code, the existing code causes error. Then I debugged with printing.

There was no key named 'transformer.wte.weight' in checkpoint.

Thank you.