Prompts with video failing with error 500

[anupdsouza]

Description of the bug:

Observing this for the past 30 hours or so. Uploading a video with a prompt is always resulting in error code 500. Image & document uploads seem to work fine. Size of the video file being 1.1 MB.

Request: ----- cURL command ----- curl -H 'x-goog-api-key: <KEY>' -H 'Content-Type: application/json' -H 'x-goog-api-client: genai-swift/0.5.4' 'https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent' -d '{"contents":[{"role":"user","parts":[{"text":"Analyze this video"},{"inline_data":{"data":"AAAAHGZ0eXBtcDQyAAAAAWlzb21tcDQxbXA0MgAAAAFtZGF0AAAAAAAGHigAACC2ZYiCAF9tD+CpeFgSic1MJlFb+kje1p6UbSkRhmtb....

Response:

[GoogleGenerativeAI] The server responded with an error: <NSHTTPURLResponse: 0x60000022ee80> { URL: https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent } { Status Code: 500, Headers {
    "Alt-Svc" =     (
        "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"
    );
    "Cache-Control" =     (
        private
    );
    "Content-Encoding" =     (
        gzip
    );
    "Content-Length" =     (
        173
    );
    "Content-Type" =     (
        "application/json; charset=UTF-8"
    );
    Date =     (
        "Fri, 16 Aug 2024 05:53:05 GMT"
    );
    Server =     (
        "scaffolding on HTTPServer2"
    );
    Vary =     (
        Origin,
        "X-Origin",
        Referer
    );
    "server-timing" =     (
        "gfet4t7; dur=7421"
    );
    "x-content-type-options" =     (
        nosniff
    );
    "x-frame-options" =     (
        SAMEORIGIN
    );
    "x-xss-protection" =     (
        0
    );
} }
[GoogleGenerativeAI] Response payload: {
  "error": {
    "code": 500,
    "message": "An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting",
    "status": "INTERNAL"
  }
}

Actual vs expected behavior:

Requests should not fail

Any other information you'd like to share?

No response

[gmKeshari]

@anupdsouza

Thank you reporting this issue. This looks like an intermittent error and should work now. This repository is for issues related to Swift SDK client bugs or improvements and the client doesn't control the service's responses. For issues related to Gemini API, we would suggest you to use "Send Feedback" option in Gemini docs. Ref: Screenshot below. You can also post this issue on Google AI forum.

[anupdsouza]

@gmKeshari, thanks. I've reported the issue to the AI Forum.

[anupdsouza]

@gmKeshari it turns out that this is a known issue with the API. According to another post, sending video data inline causes the error. The suggested way is to use the file API. Any chance the Swift SDK could be updated to support this?

[andrewheard]

@gmKeshari it turns out that this is a known issue with the API. According to another post, sending video data inline causes the error. The suggested way is to use the file API. Any chance the Swift SDK could be updated to support this?

Hi @anupdsouza, unfortunately we don't have plans to add support for uploading files with the File API at this time. Please see https://github.com/google-gemini/generative-ai-swift/issues/201#issuecomment-2278827388 for a potential alternative with the Vertex AI in Firebase SDK.

[anupdsouza]

Thanks @andrewheard. I eventually switched to Vertex as I needed video support, although it's a lot more dependencies compared to the Swift SDK. I hope video support comes around soon 🤞🏼