search

bitswired / fuseai

Self-Hosted and Open-Source web app to interact with OpenAI APIs. Currently supports ChatGPT, but DALLE and Whisper support is coming.
290 stars 45 forks source link

Utilise Dalai API for Local Llama/Alpaca AI #32

Open matteoveglia opened 1 year ago

matteoveglia commented 1 year ago

Hi, It would be great, seeing as this is a self-hosted project anyway, to have it also give the ability to connect to the dalai service created by this project: https://github.com/cocktailpeanut/dalai

The project is designed specifically for interaction with the Llama and Alpaca models, of all sizes. It has it's own web interface but it's more just for testing than actual production like ai-chat-app is.

jimzer commented 1 year ago

Thanks for sharing the tool; I didn't know it.

We could handle multiple AI APIs; I need to think of the best way to do it, though 🤔.

Maybe a configuration view, where you can specify which AI APIs you want to use and the given model parameters for the query. However, the app itself should not host the model since it's a UI to interface with AI APIs. So you would run the Llama model in a separate service and make queries via the app.

What do you think?

matteoveglia commented 1 year ago

Yeah absolutely, both this project and Dalai would remain separate, I think looking at your app as more of a 'model agnostic' portal to AI models would be the best thing absolutely.

The dalai tool allows for interfacing through a socket.io connection and it looks like you can do quite a bit with that so I think that's the best route