Open testing0mon21 opened 1 month ago
Hello @testing0mon21,
starcoder2
architecture,cohere
architecture.So from your list only llama3
is supported now.
To convert Llama 3 you have 2 options, you can do it by using Meta files and convert them by using convert-llama.py script, here is the tutorial. The second option is download .safetensor
weights from Huggingface and convert it by using convert-hf.py.
Did I understand correctly, for other architectures it will be difficult to implement the same thing that you implemented with llama? @b4rtaz
I think this depends on a specyfic architecture. Some architectures will be easy, some not. Adding new architecture is always non-zero effort. Currently DL supports: llama
, mixtral
and grok1
.
@b4rtaz Hey, thank you for your wonderful work. Could you please offer some details about how to add supported model? For example, how to to convert some ollama models like command+r or starcoder or llama3 70b to ddlama
https://ollama.com/library/command-r-plus https://ollama.com/library/llama3:70b https://ollama.com/library/starcoder2