search

nixified-ai / flake

A Nix flake for many AI projects
GNU Affero General Public License v3.0
675 stars 73 forks source link

Textgen-UI Lora support (and paths fixed) #53

Open dmadisetti opened 1 year ago

dmadisetti commented 1 year ago

Thanks for merging in the branch. The current path patch method on textgen-ui looks pretty brittle + doesn't patch LORA directory (and potentially other directories, I haven't checked)

Ideal fix for this is to get something upstream, and fix accordingly. Just opening here, but I might try to tackle this if I have time

MatthewCroughan commented 1 year ago

I think the upstream developers of both koboldai and textgen are pretty set in their ways about paths/state management. It would be an incredible refactor of their projects to do so. Personally, I'm waiting for some rust code to come along which obsoletes textgen, llama.cpp is pretty good too. I would love it if you could come up with a satisfying fix.