4bit support(ggml for cpu and gptq or gpu)?

HRNPH / AIwaifu

Open-Waifu open-sourced finetunable customizable simpable AI waifu inspired by neuro-sama

GNU General Public License v2.0

402 stars 31 forks source link

Open Jake36921 opened 1 year ago

Jake36921 commented 1 year ago

Significantly reduces ram/vram usage and faster interference.

andri-jpg commented 1 year ago

Significantly reduces ram/vram usage and faster interference.

maybe the answer will be the same as #39