search

holzschu / Carnets

Carnets is a stand-alone Jupyter notebook server and client. Edit your notebooks on the go, even where there is no network.
https://holzschu.github.io/Carnets_Jupyter/
BSD 3-Clause "New" or "Revised" License
567 stars 33 forks source link

LLM Module planned? #312

Open Ohmagar opened 1 year ago

Ohmagar commented 1 year ago

Hey there,

have you planned to integrate either of the available LLM modules? something like GPT4All, pyllama, llama.py or llama-cop-python? It would be cool to be able to integrate even the 7B models into some small scripts. Or just in general muck about.

holzschu commented 1 year ago

I'm going to think about it.

Ohmagar commented 1 year ago

Lovely to hear! Thanks for the reply.

Ohmagar commented 1 year ago

I just tested in on my mac and I underestimated how much performance it really needs. Not only did a prompt like „name the most common pets“ take about 6 minutes, other prompts were straight up garbage answers (like Name the planets in the solar system. Answer after 4 minutes „1.A“ 😂) Don’t bother with it - it’ll not be worth it Performance wise for the next few years IMO. ✌️