Open mudler opened 1 year ago
Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!
_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.
Don't engage in conversation with me, I don't support (yet) replying!
I can help you with your feature request for LocalAI. To clarify, you would like to have a rust backend added to LocalAI to support models that are supported by rustformers. This will provide an alternative to the existing C++ and Python backends. To implement this feature, we can start by adding the necessary dependencies and code to build and train the models using the rust backend. We can also ensure that the rust backend is compatible with the existing API and infrastructure of LocalAI. As for alternatives, we could consider other programming languages such as C++ or Python, but rust has several advantages over these languages in terms of performance and safety. Let me know if you have any further questions or if you would like to discuss this feature request further.
Sources:
Some notes:
Hi, guys. I am interested in this one. May I ask you guys the processing of this? I remember that we have an issue mentioned we need to have an example of building an external backend for localAI. I believe if we finish the example first and it would be more easy for us to do this. We need a template of external backend service.
@mudler May you please split the issue to a set of issues? We can pick up one by one. And it can also make sure we can do everything in the way that we want to.
we have https://github.com/go-skynet/LocalAI/issues/1057 already to track documentation on how to create LocalAI backends
Noticed and thanks.
Here is an idea of the choosing of the default backend of rust #1219
Is your feature request related to a problem? Please describe. A rust backend can be a robust alternative, and rustformers supports also other models
Describe the solution you'd like Have a rust backend to include support to models supported by rustformers
Describe alternatives you've considered
Additional context
For reference: https://github.com/huggingface/candle https://github.com/hyperium/tonic https://capnproto.org/
@lu-zero promised to give some help here =)