PDF and MP3 do not work in Gemini 1.5

[art-korn-39]

Description of the bug:

I'm trying to send mp3 and pdf files, which should be supported by the Gemini 1.5 model according to the documentation. But I always get the following response:

googleapi: Error 400: Unsupported MIME type: application/octet-stream (for .mp3) googleapi: Error 400: Unsupported MIME type: application/pdf (for .pdf)

I tried sending requests to each of two models: gemini-1.5-flash-latest gemini-1.5-pro-latest

Explicitly indicating MIME type in the genai.FileData structure didn’t help either.

Below is the request generation code (taken from the file "example_test.go")

ctx := context.Background()
client, err := genai.NewClient(ctx, option.WithAPIKey(os.Getenv("GEMINI_API_KEY")))
if err != nil {
    log.Fatal(err)
}
defer client.Close()

f, err := os.Open("path/to/file")
if err != nil {
    log.Fatal(err)
}
defer f.Close()

file, err := client.UploadFile(ctx, "", f, nil)
if err != nil {
    log.Fatal(err)
}

model := client.GenerativeModel("gemini-1.5-pro-latest")

resp, err := model.GenerateContent(ctx, genai.FileData{URI: file.URI})
if err != nil {
    log.Fatal(err)
}
_ = resp // Use resp as usual.

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[eliben]

Are you able to use/upload these files via the Google AI Studio? (https://aistudio.google.com)

[art-korn-39]

Yes, via Google AI Studio it works good.

[andrewheard]

Hi @art-korn-39, I don't think PDFs are a supported image format yet but MP3s should work if the MIME type is set to audio/mp3 instead of application/octet-stream (see audio formats).

@eliben I tried setting application/octet-stream in the Swift SDK and was able to reproduce the error above for MP3s but it worked fine with audio/mp3.

[art-korn-39]

Yes, I confirm, it works with mp3, the main thing is to specify the MIME type when uploading the file. Thanks!

Sample: file, err := gen_client.UploadFile(gen_ctx, "", f, &genai.UploadFileOptions{MIMEType: "audio/mp3"})