search

getumbrel / umbrel-apps

The official app repository of the Umbrel App Store. Submit apps and updates here. Learn how → https://github.com/getumbrel/umbrel-apps#readme
https://apps.umbrel.com
481 stars 356 forks source link

Add localai #1079

Open highghlow opened 1 month ago

highghlow commented 1 month ago

noGPUpasthu4u

App Submission

App name

LocalAI

256x256 SVG icon

(Submit an icon with no rounded corners as it will be dynamically rounded with CSS. GitHub doesn't allow uploading SVGs directly, so please upload your icon to an alternate service, like https://svgur.com, and paste the link below.) We will help finalize this icon before the app goes live in the Umbrel App Store.

For some reason I was only able to find this as a jpeg

https://github.com/go-skynet/LocalAI/assets/2420543/0966aa2a-166e-4f99-a3e5-6c915fc997dd

Gallery images

(Upload 3 to 5 high-quality gallery images (1440x900px) of your app in PNG format, or just upload 3 to 5 screenshots of your app and we'll help you design the gallery images.) We will help finalize these images before the app goes live in the Umbrel App Store. ...

Screenshot 2024-05-08 at 08-44-05 LocalAI API - v2 14 0 (b58274b8a26a3d22605e3c484cf39c5dd9a5cf8e)

image

I can't get more screenshots because running this without gpu passthrough is really, really slow.

I have tested my app on:

nmfretz commented 1 month ago

Thanks for getting this together @highghlow. I've been playing around with it this morning to package it so that it installs on a users machine with no/minimal models downloaded initially. The image and the models are quite large, so install time is very long.

But I've just realized embarrassingly late in my testing that the localai images are not multi-architecture. They are only available for x86 right now. This means that the install would fail for arm users (e.g., RPi 5) and leave users in a broken state. Let's close this for now until an official arm image is available. I will open an Issue in their repo asking about official arm support.

nmfretz commented 1 month ago

Ohhhhh, hold up I see that arm support is on the horizon so let's keep this PR open! https://github.com/mudler/LocalAI/issues/1249#issuecomment-2134721490

Once they are available I'm hoping we can use their "standard container images" which don't include pre-installed models: https://localai.io/basics/container/#standard-container-images