feature: rust https://github.com/rustformers backend

[mudler]

Is your feature request related to a problem? Please describe. A rust backend can be a robust alternative, and rustformers supports also other models

Describe the solution you'd like Have a rust backend to include support to models supported by rustformers

Describe alternatives you've considered

Additional context

For reference: https://github.com/huggingface/candle https://github.com/hyperium/tonic https://capnproto.org/

@lu-zero promised to give some help here =)

[localai-bot]

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

I can help you with your feature request for LocalAI. To clarify, you would like to have a rust backend added to LocalAI to support models that are supported by rustformers. This will provide an alternative to the existing C++ and Python backends. To implement this feature, we can start by adding the necessary dependencies and code to build and train the models using the rust backend. We can also ensure that the rust backend is compatible with the existing API and infrastructure of LocalAI. As for alternatives, we could consider other programming languages such as C++ or Python, but rust has several advantages over these languages in terms of performance and safety. Let me know if you have any further questions or if you would like to discuss this feature request further.

Sources:

• https://localai.io/howtos/index.html
• https://localai.io/advanced/index.html
• https://github.com/go-skynet/LocalAI/tree/master/.github/ISSUE_TEMPLATE/feature_request.md
• https://localai.io/features/index.html

[lu-zero]

Some notes:

• if LocalAI wants to explore capnproto to replace the protobuf interface that is orthogonal to supporting rust backends.
• burn might be also interesting to consider since it may or may not support additional backends, while integrating candle examples might help us to compare quickly with already existing features.

[Aisuko]

Hi, guys. I am interested in this one. May I ask you guys the processing of this? I remember that we have an issue mentioned we need to have an example of building an external backend for localAI. I believe if we finish the example first and it would be more easy for us to do this. We need a template of external backend service.

@mudler May you please split the issue to a set of issues? We can pick up one by one. And it can also make sure we can do everything in the way that we want to.

[mudler]

we have https://github.com/go-skynet/LocalAI/issues/1057 already to track documentation on how to create LocalAI backends

[lu-zero]

Here is the equivalent tutorial but using tonic.

[Aisuko]

Noticed and thanks.

[Aisuko]

Here is an idea of the choosing of the default backend of rust #1219