Llama v1 import from HF support

epfLLM / Megatron-LLM

distributed trainer for LLMs

Other

504 stars 73 forks source link

Closed AleHD closed 10 months ago

AleHD commented 10 months ago

Features:

Support for converting huggingface llama v1 weights (from decapoda-research/llama-7b-hf) to megatron, thus providing easier access to llama v1.
Support for testing correct llama v1 weight conversion (implemented in tests/test_llama_weights.py

AleHD commented 10 months ago

TODO before merging:

[x] Running locally test_llama_weights.py to verify that the merge with main was performed successfully.

martinjaggi commented 10 months ago

need to rebase now that converters are refactored

AleHD commented 10 months ago

TODO before merging:

[x] Running locally test_llama_weights.py to verify that the merge with main was performed successfully.