hello [Feature] 🌈 vision models doesnt follow openai api schema

[userbox020]

Describe the problem related to the feature request

im getting the follow error when trying to ask questions about images

open-webui  | ERROR:apps.openai.main:0, message='Attempt to decode JSON with unexpected mimetype: text/plain; charset=utf-8', url=URL('http://192.168.1.83:8082/v1/chat/completions')
open-webui  | Traceback (most recent call last):
open-webui  |   File "/app/backend/apps/openai/main.py", line 469, in generate_chat_completion
open-webui  |     response_data = await r.json()
open-webui  |                     ^^^^^^^^^^^^^^
open-webui  |   File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1176, in json
open-webui  |     raise ContentTypeError(
open-webui  | aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/plain; charset=utf-8', url=URL('http://192.168.1.83:8082/v1/chat/completions')

Uh-oh! There was an issue connecting to Precise-vision.
text input must be of type `str` (single example), `List[str]` (batch or single pretokenized example) or `List[List[str]]` (batch of pretokenized examples).

Describe the solution you'd like

upload images and ask about them

Describe alternatives you've considered

ollama vision models working fine

Additional context

No response

[Harry-zklcdc]

It's fine on NextChatWeb, please Check Your Code

[userbox020]

@Harry-zklcdc hello and thanks for your time and response, the post that open webui does https://github.com/open-webui/open-webui is the follow

POST /openai/chat/completions HTTP/1.1
Host: localhost:80
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Authorization: Bearer $Auth-Token
Content-Type: application/json
Origin: http://localhost:80
Connection: keep-alive
Priority: u=4
Accept-Encoding: gzip, deflate
Content-Length: 2774263

{"model":"Precise-vision","stream":true,"stream_options":{"include_usage":true},"messages":[{"role":"user","content":[{"type":"text","text":"describe the image"},{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,/9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAZAAA/...(VERY LONG STRING...)4/KoP8ru/wCn7nvf/wCJv2bf/wChjf8A/VZ07/vbO1r/AMK5f/4X1X9e7/6Hpb8a/9k="}}]}],"citations":false,"chat_id":"cf2e96eb-f693-4e67-822c-1a271fb51dd1"}

and the response it get back is the follow:

HTTP/1.1 500 Internal Server Error
date: Fri, 28 Jun 2024 23:54:19 GMT
server: uvicorn
content-type: application/json
access-control-allow-origin: *
access-control-allow-credentials: true
x-process-time: 0
Content-Length: 169

{"detail":"text input must be of type `str` (single example), `List[str]` (batch or single pretokenized example) or `List[List[str]]` (batch of pretokenized examples)."}