Hey everyone! π I'm super excited to share what we've been working on at LocalAI - the launch of v2.11.0. This isn't just any update; it's a massive leap forward, making LocalAI easier to use, faster, and more accessible for everyone.
π The Spotlight: All-in-One Images, OpenAI in a box
Imagine having a magic box that, once opened, gives you everything you need to get your AI project off the ground with generative AI. A full clone of OpenAI in a box. That's exactly what our AIO images are! Designed for both CPU and GPU environments, these images come pre-packed with a full suite of models and backends, ready to go right out of the box.
Whether you're using Nvidia, AMD, or Intel, we've got an optimized image for you. If you are using CPU-only you can enjoy even smaller and lighter images.
To start LocalAI, pre-configured with function calling, llm, tts, speech to text, and image generation, just run:
docker run -p 8080:8080 --name local-ai -ti localai/localai:latest-aio-cpu
Do you have a Nvidia GPUs? Use this instead
CUDA 11
docker run -p 8080:8080 --gpus all --name local-ai -ti localai/localai:latest-aio-gpu-cuda-11
CUDA 12
docker run -p 8080:8080 --gpus all --name local-ai -ti localai/localai:latest-aio-gpu-cuda-12
:heart: Why You're Going to Love AIO Images:
Ease of Use: Say goodbye to the setup blues. With AIO images, everything is configured upfront, so you can dive straight into the fun part - hacking!
Flexibility: CPU, Nvidia, AMD, Intel? We support them all. These images are made to adapt to your setup, not the other way around.
Speed: Spend less time configuring and more time innovating. Our AIO images are all about getting you across the starting line as fast as possible.
π Jumping In Is a Breeze:
Getting started with AIO images is as simple as pulling from Docker Hub or Quay and running it. We take care of the rest, downloading all necessary models for you. For all the details, including how to customize your setup with environment variables, our updated docs have got you covered here, while you can get more details of the AIO images here.
π Vector Store
Thanks to the great contribution from @βrichiejp now LocalAI has a new backend type, "vector stores" that allows to use LocalAI as in-memory Vector DB (mudler/LocalAI#1792). You can learn more about it here!
π Bug fixes
This release contains major bugfixes to the watchdog component, and a fix to a regression introduced in v2.10.x which was not respecting --f16, --threads and --context-size to be applied as model's defaults.
π New Model defaults for llama.cpp
Model defaults has changed to automatically offload maximum GPU layers if a GPU is available, and it sets saner defaults to the models to enhance the LLM's output.
π§ New pre-configured models
You can now run llava-1.6-vicuna, llava-1.6-mistral and hermes-2-pro-mistral, see Run other models for a list of all the pre-configured models available in the release.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Bumps go-skynet/local-ai from v2.9.0-ffmpeg-core to v2.11.0-ffmpeg-core.
Release notes
Sourced from go-skynet/local-ai's releases.
... (truncated)
Commits
1395e50
:arrow_up: Update ggerganov/llama.cpp (#1897)42a4c86
:arrow_up: Update ggerganov/whisper.cpp (#1896)c9adc56
fix(aio): make image-gen for GPU functional, update docs (#1895)08c7b17
Fix NVIDIA VRAM detection on WSL2 environments (#1894)5e12382
NVIDIA GPU detection support for WSL2 environments (#1891)6cf9952
docs(aio): Add All-in-One images docs (#1887)3e293f1
:arrow_up: Update ggerganov/llama.cpp (#1889)0106c58
:arrow_up: Update ggerganov/llama.cpp (#1885)bd25d80
fix(watchdog): use ShutdownModel instead of StopModel (#1882)49cec7f
ci(aio): add latest tag images (#1884)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show