Open fkiifdjo opened 1 year ago
The slow loading happens every time a new chat message is being generated, it's particularly noticeable with 30b models but also noticeable with 13b.
There are currently two issues going on:
So on the Kobold side we are waiting for either GPTQ package to get the needed updates before the full speed is restored.
The slow loading happens every time a new chat message is being generated, it's particularly noticeable with 30b models but also noticeable with 13b.