search

edubruell / tidyllm

An tidy interface to large language model APIs for R
https://edubruell.github.io/tidyllm/
Other
36 stars 2 forks source link

Article on working with audio/video files in Gemini API #30

Closed edubruell closed 1 week ago

edubruell commented 2 weeks ago

Add an article demonstrating how to work with media files (video/audio) using the Gemini API in tidyllm, highlighting its unique ability to handle video uploads in API calls.

Expand the small example from the newslog on Gemini:

#Upload a file for use with gemini
upload_info <- gemini_upload_file("example.mp3")

#Make the file available during a Gemini API call
llm_message("Summarize this speech") |>
  gemini(.fileid = upload_info$name)

#Delte the file from the Google servers
gemini_delete_file(upload_info$name)

Proposed Content:

  1. Introduction:

    • Overview of the Gemini API's media handling capabilities.
    • Highlight its support for a wide range of audio and video formats.

  2. Practical Example:

    • Upload a video file using gemini_upload_file().
    • Use the uploaded video as context in an API call (gemini()).
    • Demonstrate extracting insights from video (e.g., summarization, sentiment analysis).

  3. File Management:

    • Show how to manage uploaded files with gemini_list_files()and gemini_delete_file().
    • Explain the lifecycle of media files in Gemini workflows.