Localai v2 Image2Image not working

LocalAI version:

Dockerimage quay.io/go-skynet/local-ai:v2.0.0-cublas-cuda11-ffmpeg Describe the bug

I'm trying to implement an Image2Image transformation using the Stable Diffusion model with Diffusers, as described in the guide https://localai.io/model-compatibility/diffusers/. I am using the latest Docker image v2.0.0. Normal Text-to-Image generation is working. However, I keep encountering an error, and I'm not sure what I'm doing wrong. Here's the error message I receive:

{
   "error":{
      "code":500,
      "message":"rpc error: code = Unknown desc = Exception calling application: 'image'",
      "type":""
   }
}

I have set up my YAML configuration as follows:

backend: diffusers
diffusers:
  cuda: true
  enable_parameters: negative_prompt,num_inference_steps,image 
  pipeline_type: StableDiffusionImg2ImgPipeline
f16: true
name: stablediffusion-edit
parameters:
  model: nitrosocke/Ghibli-Diffusion
step: 25

And here's the Python script I'm using to encode the image in Base64 and create the JSON request:

import base64
import json

# Convert image to Base64
with open("test.jpg", "rb") as image_file:
    encoded_string = base64.b64encode(image_file.read()).decode()

# Create JSON body
data = {
    "image": encoded_string,
    "prompt": "a sky background",
    "size": "512x512",
    "model": "stablediffusion-edit"
}

# Save JSON to file
with open('request.json', 'w') as outfile:
    json.dump(data, outfile)

# Execute cURL command
!curl -H "Content-Type: application/json" -d @request.json http://localhost:8080/v1/images/generations

Also tried it with Openai:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="sk-xxx")

client.images.edit(
  image=open("original.png", "rb"),
  mask=open("mask.png", "rb"),
  prompt="A cute baby sea otter wearing a beret",
  n=2,
  size="256x256"
)

Here I am getting the error:

NotFoundError: Error code: 404 - 
{
   "error":{
      "code":404,
      "message":"Cannot POST /v1/images/edits",
      "type":""
   }
}

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

ERROR: The prompt size exceeds the context window size and cannot be processed.

Sources:

mudler / LocalAI

Localai v2 Image2Image not working #1392

:warning::warning::warning::warning::warning:

:warning::warning::warning::warning::warning: