Image generation is not working - 500 internal server error

[Killerjoe]

Which version of integration_openai are you using?

1.1.4

Which version of Nextcloud are you using?

27.1.4

Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.

Firefox

Describe the Bug

Hi, I try to get Image Generation with installed Local AI working. Text generation is working but image generation not. I go into collectives and type slash image generation with Local aI and type for example pizza in prompt. and get back this error: "API request error: API request error: failed reading parameters from request:failed parsing request body: json: cannot unmarshal string into Go struct field OpenAIRequest.response_format of type schema.ChatCompletionResponseFormat" If I try to run the image generation from shell, it is working with this curl:

curl -X POST http://localhost:8080/v1/images/generations -H "Content-Type: application/json" -d '{"prompt":"pizza","size":"256x256"} {"created":1701806913,"id":"b995a438-ed67-40de-ba90-8ced26054aed","data":[{"embedding":null,"index":0,"url":"http://localhost:8080/generated-images/b642891034205.png"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}' I could not find somebody else with this error, so I have to raise an issue.

Regards, Joe

Expected Behavior

Image generation should work with Local AI Image generation.

To Reproduce

Go into collectives in a site, use / and type image generation with Local Ai. Type pizza in prompt and proceed with Vorschau/Preview. Error will be shown top right.

[MB-Finski]

LocalAI doesn't probably like that the model key is now specified for openAI. I'll look into this.

EDIT: Yeah, seems like LocalAI has not yet matched the new features in the openAI api.

[Killerjoe]

Hi MB-Finski, I have forgot to mention that this issue was already there in version 1.1.2 of the integration_openai app. Thanks.

[MB-Finski]

Ah.. Ok, so this has actually changed in LocalAI at some point. It now rejects all the other parameters also that were previously defined for OpenAI. I.e. the API is no longer really OpenAI compatible. So we'll have to make different requests for either.

[ER-EPR]

Same here, it was OK yesterday. But after I update both LocalAI from 1.40.0 to 2.0.0, and integration_openai to 1.1.4, image generation fail to work. Error message is the same as above.

[MB-Finski]

Yeah, it's a breaking change in LocalAI. I have the necessary changes for the integration backend already in testing.

[ER-EPR]

That's Great! Looking forward to the update. Will it be released in a few days, so I won't have to downgrade LocalAI?

[MB-Finski]

Should be fixed now in v1.1.5. Could you, please, test that the fix also works with your setup and report back?

[Killerjoe]

Thank you very much MB-Finski for your fast help and support, it is working now in my setup with version 1.1.5 :-)