[Bug]: vision not working for gpt-3.5-turbo and gpt-4o-mini

[Znunu]

What happened?

I am using chatGPT with vision through litellm. it works fine with gpt-4o, however it doesn't work with gpt-3.5-turbo and gpt-4o-mini.

example

this is an example of the messages I used to test with

[{'type': 'text', 'text': 'zun said " look at this!"'}, 
{'type': 'image_url', 
'image_url': {'url': 'https://cdn.discordapp.com/attachments/1231985772941410386/1263802670397915227/image.png?ex=669b8f74&is=669a3df4&hm=0fd2cce577a8804df392a700521f71b37f72c7b6e8b02412d120f49e6909a54e&', 
'detail': 'low'}}]

gpt-3.5-turbo

seemingly it complains that I didn't give it a base64 image, instead of an image link. however I should be allowed to just give an image link

OpenAIException - Error code: 400 - {'error': {'message': "Invalid image URL: 'messages[7].content[1].image_url.url'. Expected a base64-encoded data URL with an image MIME type (e.g. 'data:image/png;base64,aW1nIGJ5dGVzIGhlcmU='), but got a value without the 'data:' prefix."

gpt-4o-mini

sort of works, but it uses 3k!!! tokens even when uploading the tiniest image and with "detail: low"

Relevant log output

No response

Twitter / LinkedIn details

No response

[Clad3815]

• gpt3.5 = No Vision
• gpt-4o-mini = Same price than gpt-4o for Vision, so it's use more tokens to adjust the price. Check on the OpenAI pricing page

[Znunu]

thanks a lot! that clears it up