search

openppl-public / ppl.llm.serving

Apache License 2.0
122 stars 13 forks source link

support for huggingface llama? #16

Closed sleepwalker2017 closed 9 months ago

sleepwalker2017 commented 10 months ago

We use vicuna in huggingface format. I notice only meta format is supported ? Is that true?

Alcanderian commented 10 months ago

The script for converting huggingface format to meta format will be released ASAP in https://github.com/openppl-public/ppl.pmx

sleepwalker2017 commented 10 months ago

The script for converting huggingface format to meta format will be released ASAP in https://github.com/openppl-public/ppl.pmx

thank you, got it.

Vincent-syr commented 9 months ago

Here is the exporting script for llama hugging face https://github.com/openppl-public/ppl.pmx/tree/master/model_zoo/llama/huggingface