search

EvanZhouDev / gemini-ai

The simpler JavaScript Gemini SDK
https://www.npmjs.com/package/gemini-ai
GNU General Public License v3.0
95 stars 16 forks source link

Media type detection not working #17

Open XInTheDark opened 2 weeks ago

XInTheDark commented 2 weeks ago

Currently only images are detected correctly. Video formats (like .mov) or plain text (.txt) documents are erroneously rejected.

Steps to reproduce:

  1. let buffer = fs.readFileSync("path/to/any/mov or txt file")
  2. pass buffer to the API as a file upload

The API will return an error saying "Error: Please provide a valid file format that is accepted by Gemini. Learn more about valid formats here: https://ai.google.dev/gemini-api/docs/prompting_with_media?lang=node#supported_file_formats".

Expected behaviour: The API should accept all these video and plaintext formats.

XInTheDark commented 2 weeks ago

I have done some debugging and the issue is because fileTypeFromBuffer (used in utils.ts) wrongly returns undefined on txt, mov, etc. files.

XInTheDark commented 2 weeks ago

A possible fix would be to fall back to using the file extension if the MIME type not found, using https://www.npmjs.com/package/mime (lite version) which is really compact.

EvanZhouDev commented 6 days ago

The issue is, Gemini AI is not in any way aware of the file extension (or even the file name at all, for that matter). This is by design, as Gemini AI was not supposed to read from files directly anyway (as that would require fs, thus involving a Node-only dep, causing browser incompatibility. In the past, we have considered dynamically importing fs, but that was just a mess)... however, what I can do is make it so that the user can manually specify a MIME type.