search

pytorch / executorch

On-device AI across mobile, embedded and edge for PyTorch
https://pytorch.org/executorch/
Other
1.41k stars 230 forks source link

[FEATURE REQUEST] GGUF inputs #4006

Closed rcontesti closed 1 week ago

rcontesti commented 1 week ago

Llama.cpp has proven quite convininent for many of us, I wish you could add the possibility of taking those models as inputs. Many thanks!

mergennachin commented 1 week ago

@rcontesti - we have a work-in-progress prototype to convert fp16/fp32 version of llama

https://github.com/pytorch/executorch/tree/main/extension/gguf_util

rcontesti commented 1 week ago

Once again many thanks @mergennachin