Description: Enhance the llm_message() function to support attaching multiple images per message and accept both local image files and remote URLs instead of just base64-encoded local images. This aligns the function with APIs like Mistral and OpenAI that allow multiple images and support both base64-encoded images and remote URLs. Additionally, implement a fallback mechanism for APIs that do not support remote URLs (e.g., local Ollama) by downloading images and converting them to base64 format.
Possible Actions:
Replace the .imagefile parameter with an .image-paramater that accepts a vector of image paths or URLs:
Keep .imagefile parameter in function as wrapper around new parameter and display deprecation message.
Update image handling to distinguish between local files and remote URLs:
Implement logic to check if each entry in .image is a local file path or a remote URL.
For local files:
Validate the file type (e.g., JPEG, PNG).
Base64 encode the image as before.
For remote URLs:
Validate the URL format.
Add the URL directly to the media_list if the target API supports it.
Implement a fallback mechanism for APIs that do not support remote URLs:
Use wget or a similar approach to download the image from a provided URL.
Encode the downloaded image as base64 and attach it to the media_list.
Apply this fallback mechanism selectively, based on the capabilities of the target API in to_api_format()-method.
Adjust the media_list structure to handle multiple images:
Ensure that media_list can store multiple image entries, each as a separate list item.
Include a source field to indicate whether the image is from a local file, remote URL, or a downloaded base64-encoded file.
Update documentation and provide examples:
Revise the function's documentation to explain the new parameters and capabilities.
Include examples demonstrating how to attach multiple images, use remote URLs, and fall back to base64 encoding.
Add error handling and validation:
Implement checks to ensure that provided URLs are valid and accessible.
Provide informative error messages if an image fails to load, download, or encode.
Description: Enhance the
llm_message()function to support attaching multiple images per message and accept both local image files and remote URLs instead of just base64-encoded local images. This aligns the function with APIs like Mistral and OpenAI that allow multiple images and support both base64-encoded images and remote URLs. Additionally, implement a fallback mechanism for APIs that do not support remote URLs (e.g., local Ollama) by downloading images and converting them to base64 format.Possible Actions:
Replace the
.imagefileparameter with an.image-paramater that accepts a vector of image paths or URLs:.imagefileparameter in function as wrapper around new parameter and display deprecation message.Update image handling to distinguish between local files and remote URLs:
.imageis a local file path or a remote URL.media_listif the target API supports it.Implement a fallback mechanism for APIs that do not support remote URLs:
wgetor a similar approach to download the image from a provided URL.media_list.to_api_format()-method.Adjust the
media_liststructure to handle multiple images:media_listcan store multiple image entries, each as a separate list item.sourcefield to indicate whether the image is from a local file, remote URL, or a downloaded base64-encoded file.Update documentation and provide examples:
Add error handling and validation: