How to compile LocalAI with stablediffusion on Docker container

[Soberia]

LocalAI version: v1.13.0 commit ffaf3b1d362cc4d0045e1ed60bbcf0481d21f8ef

Describe the bug I changed make build to make GO_TAGS=stablediffusion build in Dockerfile and during the build process, I can see in the logs that the github.com/mudler/go-stable-diffusion repository gets cloned but whenever I make an API call to the /v1/images/generations endpoint, I always get this error:

This version of LocalAI was built without the stablediffusion tag

directory content:

root@ai:/home/ai/LocalAI/models/stablediffusion_assets# du -ahd1
95M ./AutoencoderKL-fp16.bin
60K ./UNetModel-256-MHA-fp16.param
470M    ./FrozenCLIPEmbedder-fp32.onnx
190M    ./AutoencoderKL-fp32.onnx
1.7G    ./UNetModel-MHA-fp16.bin
104K    ./UNetModel-fp16.param
36K ./FrozenCLIPEmbedder-fp16.param
66M ./AutoencoderKL-encoder-512-512-fp16.bin
20K ./AutoencoderKL-512-fp16-opt.param
236M    ./FrozenCLIPEmbedder-fp16.bin
76K ./UNetModel-512-MHA-fp16-opt.param
56K ./UNetModel-512-MHA-fp16.param
16K ./AutoencoderKL-512-fp16.param
20K ./AutoencoderKL-256-fp16-opt.param
16K ./AutoencoderKL-256-fp16.param
1.7G    ./UNetModel-fp16.onnx
76K ./UNetModel-256-MHA-fp16-opt.param

[ralyodio]

https://github.com/go-skynet/LocalAI#build-with-image-generation-support

but i donj't know how to run this command.

[ralyodio]

I think you have to change BUILD_TYPE in .env but i'm not sure to what.

[ralyodio]

I just tried what you did originally and i canj't connect:

$ curl http://localhost:9220/v1/models
curl: (7) Failed to connect to localhost port 9220 after 0 ms: Couldn't connect to server

[Soberia]

I think you have to change BUILD_TYPE in .env but i'm not sure to what.

I tried with BUILD_TYPE=openblas before, no luck either.

[mudler]

This is a lack in our docs. Did you tried by setting GO_TAGS=stablediffusion in the .env file?

[ralyodio]

i still get {"error":{"code":500,"message":"This version of LocalAI was built without the stablediffusion tag","type":""}}

after add GO_TAGS to .env

[mudler]

i still get {"error":{"code":500,"message":"This version of LocalAI was built without the stablediffusion tag","type":""}}

after add GO_TAGS to .env

How are you running LocalAI? Do you set also REBUILD=false by any chance? default binaries in the image don't have stable diffusion enabled, but I think it's safe to enable at this point.

[ralyodio]

no i have REBUILD=true

# THREADS=14
# CONTEXT_SIZE=512
MODELS_PATH=/models
DEBUG=true
#BUILD_TYPE=generic
REBUILD=true
PORT=9220
GO_TAGS=stablediffusion

[Soberia]

I commented if statement in Makefile to make sure required packages are compiled at the build time. I think this error is not related to the presence of stablediffusion. I don't know the Go language but is this function throws this error for any failure?

[lenaxia]

Did you end up getting this working? Trying to figure out how to do this for kubernetes

Edit: Adding GO_TAGS: stablediffiusion to the container environment variables puts the container into a crash loop

[mudler]

Did you end up getting this working? Trying to figure out how to do this for kubernetes

Edit: Adding GO_TAGS: stablediffiusion to the container environment variables puts the container into a crash loop

can you take the logs and file a separate issue for this? I'm running this in Kubernetes and don't observe any crashloop here

[mudler]

no i have REBUILD=true

# THREADS=14
# CONTEXT_SIZE=512
MODELS_PATH=/models
DEBUG=true
#BUILD_TYPE=generic
REBUILD=true
PORT=9220
GO_TAGS=stablediffusion

Can you share how you start LocalAI and the full logs?

a make GO_TAGS=stablediffusion build here yields:

base ❯ make GO_TAGS=stablediffusion build                                                                                                                                                                                                    
[...]
I local-ai build info:
I BUILD_TYPE: 
I GO_TAGS: stablediffusion
CGO_LDFLAGS="" C_INCLUDE_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/gpt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwk
v:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz LIBRARY_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/g
pt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwkv:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz go build -tags "stablediffusion" -x -o l
ocal-ai ./
WORK=/tmp/go-build1217960430

[Soberia]

a make GO_TAGS=stablediffusion build here yields:

base ❯ make GO_TAGS=stablediffusion build                                                                                                                                                                                                    
[...]
I local-ai build info:
I BUILD_TYPE: 
I GO_TAGS: stablediffusion
CGO_LDFLAGS="" C_INCLUDE_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/gpt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwk
v:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz LIBRARY_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/g
pt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwkv:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz go build -tags "stablediffusion" -x -o l
ocal-ai ./
WORK=/tmp/go-build1217960430

@mudler Thanks for the hint. I think I found the problem. Here's the log when GO_TAGS=stablediffusion is set in .env:

When I change the Dockerfile.dev make command to:

sed -i 's/make build/make GO_TAGS=stablediffusion build/' Dockerfile.dev

eventually, stablediffusion will be compiled:

I also tried to modify the dockerfile manually before but I was looking for the wrong one. The Dockerfile.dev is referenced in compose file and not the Dockerfile. (why?)

So I think to fix this issue, in either Dockerfile or Dockerfile.dev, RUN make build should be changed to RUN make $GO_TAGS build to read the environment variable correctly for the build process.

[mudler]

a make GO_TAGS=stablediffusion build here yields:

base ❯ make GO_TAGS=stablediffusion build                                                                                                                                                                                                    
[...]
I local-ai build info:
I BUILD_TYPE: 
I GO_TAGS: stablediffusion
CGO_LDFLAGS="" C_INCLUDE_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/gpt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwk
v:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz LIBRARY_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/g
pt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwkv:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz go build -tags "stablediffusion" -x -o l
ocal-ai ./
WORK=/tmp/go-build1217960430

@mudler Thanks for the hint. I think I found the problem. Here's the log when GO_TAGS=stablediffusion is set in .env:

When I change the Dockerfile.dev make command to:

sed -i 's/make build/make GO_TAGS=stablediffusion build/' Dockerfile.dev

eventually, stablediffusion will be compiled:

I also tried to modify the dockerfile manually before but I was looking for the wrong one. The Dockerfile.dev is referenced in compose file and not the Dockerfile. (why?)

So I think to fix this issue, in either Dockerfile or Dockerfile.dev, RUN make build should be changed to RUN make $GO_TAGS build to read the environment variable correctly for the build process.

how are you running it in docker-compose? try to pull the image and specify --pull always instead of --build. --build will use Dockerfile.dev which is used for development and doesn't recompile on start.

[Soberia]

how are you running it in docker-compose?

git clone https://github.com/go-skynet/LocalAI
cd LocalAI && docker compose build

[mudler]

how are you running it in docker-compose?

git clone https://github.com/go-skynet/LocalAI
cd LocalAI && docker compose build

Try with:

git clone https://github.com/go-skynet/LocalAI
cd LocalAI && docker compose up --pull always

[Soberia]

@mudler The GO_TAGS environment variable is set correctly when the image builds with docker compose up instead of docker compose build and there is no need to modify the dockerfile manually.

Thank you. I close this now.