search

lmstudio-ai / configs

LM Studio JSON configuration file format and a collection of example config files.
MIT License
172 stars 26 forks source link

Gemma models still not working. #21

Open rombodawg opened 4 months ago

rombodawg commented 4 months ago

Gemma models that have been quantized using Llamacpp are not working. Please look into the issue

error

"llama.cpp error: 'create_tensor: tensor 'output.weight' not found'"

I will open a issue on the llamacpp github aswell addressing this

https://github.com/ggerganov/llama.cpp/issues/5706

System: Ryzen 5600x rtx 3080 gpu b550 motherboard 64gb ddr4 ram windows 10 OS

rombodawg commented 4 months ago

Im uploading the model files for the merges if anyone wants to do some debugging. Should be in the next 10 hours or so. Sorry slow internet.

Follow the mulit-thread. And check out my model for debugging.

Thread links: https://github.com/lmstudio-ai/configs/issues/21 https://github.com/ggerganov/llama.cpp/issues/5706 https://github.com/arcee-ai/mergekit/issues/181 https://github.com/oobabooga/text-generation-webui/issues/5562

https://huggingface.co/rombodawg/Gemme-Merge-Test-7b