holtskinner
commented
3 weeks ago You can pass in multiple types of data, such as Images/Video/Audio. Example: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/image-understanding
And you can send a tokenized prompt as a list of strings like this:
import vertexai
from vertexai.generative_models import GenerativeModel
# TODO(developer): Update project_id
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")
model = GenerativeModel("gemini-1.5-flash-001")
response = model.generate_content(
["What", "'s", " a", " good", " name", " for", " a", " flower", " shop", " that", " specializes", " in", " selling", " bou", "quets", " of", " dried", " flowers", "?"]
)
print(response.text)
Is your feature request related to a problem? Please describe.
When using the REST api, the json object has "text": "this is a prompt" fields. Instead of text what other types are possible? it would be great to be able to send already tokenized data (it would speed up the inference and save network data too).
Describe the solution you'd like
a more efficient way to send prompts and history or chubnked prompts.
Describe alternatives you've considered
No response
Additional context
No response
Code of Conduct