Closed jexp closed 4 months ago
Perhaps in addition we should add a generic URL method like we did in bedrock that allows to use more custom new model URLs
and perhaps an Image Vision procedure as well as we did for Bedrock.
https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/image-generation
it seems to support a GCS storage URL as optional parameter, so that the generated images are stored in GCS and an URL returned, we should support that.
And also see if something similar is available for the bedrock image-gen API with S3 and add it there too.
see:
currently we use
:predict
as suffix which doesn't seem to work anymore with gemini, also the request body changed a bitwe can probably have a new method called apoc.ml.vertexai.stream
that also works with the new body.
gemini-pro-vision supports also image payloads to e.g. describe or summarize images, not sure if we need that yet
https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini#request_body
POST https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/publishers/google/models/gemini-pro-vision:streamGenerateContent
https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini#gemini-pro-vision
We probably need that for gemini ultra anyhow.