TripSit / TripBot

The discord bot on TripSit.Me
24 stars 13 forks source link

Move over to self-hosted models via local AI or olama? #856

Open AlanaRm-rf-me opened 1 month ago

AlanaRm-rf-me commented 1 month ago

With the growth around being able to make rag AI easier and cheaper to run on hardware I was thinking if we were to have chat gpt that's already iffy enough when it comes to harm reduction where it's just kind of a novelty in my opinion to have it. What are your thoughts? As I believe we should start looking into the locally posted model about can be ran on our servers and completely customized with the data trips it has built up

There's a few models that I have in mind the main one would be llama 3.2 but there is one specifically that I believe would benefit us the best I'll comment and update if I end up finding the name of it.

But local AI is also an option as we can actually do GPT I believe gpt for all I believe is the name several options to look intotakeI work on the website and mainly as you can tell trying out the action workflows on GitHub and failing massively so I thought to make a discussion post and let y'all decide if this is the route to take