search

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
https://localai.io
MIT License
23.86k stars 1.83k forks source link

unify transformer-based backends #1411

Open mudler opened 10 months ago

mudler commented 10 months ago

There are few backends based of transformers that can be unified into a single one. This would help to reduce the image size and CI build times

localai-bot commented 10 months ago

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

I suggest creating a new backend that combines the features of the existing backends and optimizes it for the specific use case of music generation. This could potentially improve performance and reduce the image size. To debug the issue with the transformers project, I recommend checking the logs for any errors or issues during the build process. Additionally, you can try building the project on a different machine or using a different version of the tools to see if that resolves the issue. If the issue persists, it may be helpful to file a bug report with the transformers project team.

Sources: