Question | Has anyone tried this with GGUF models?

datamllab / LongLM

[ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

https://arxiv.org/pdf/2401.01325.pdf

MIT License

597 stars 59 forks source link

Question | Has anyone tried this with GGUF models? #32

Open Cognitus-Stuti opened 5 months ago

Mooler0410 commented 5 months ago

Llama.cpp has supported SelfExtend and had a good implementation. It uses GGUF models. SelfExtend has obtained pretty positive feedback from Llama.cpp's community. You can check their repo for more details.

piotr25691 commented 1 month ago

probably, but i'd recommend to just make a .safetensors file Self-Extended before making it GGUF again