Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in Holistic Evaluation of Text-to-Image Models (HEIM) (https://arxiv.org/abs/2311.04287).
Gemini-1.5-Pro-002 and Gemini-1.5-Flash-002 should work exactly the same as the older snapshots google/gemini-1.5-pro-001 and google/gemini-1.5-flash-001, so it's a matter of copy and pasting the model metadata /deployment and updating the strings.
liamjxu
commented
4 days ago
Blog: https://developers.googleblog.com/en/updated-production-ready-gemini-models-reduced-15-pro-pricing-increased-rate-limits-and-more/
Gemini-1.5-Pro-002 and Gemini-1.5-Flash-002 should work exactly the same as the older snapshots
google/gemini-1.5-pro-001andgoogle/gemini-1.5-flash-001, so it's a matter of copy and pasting the model metadata /deployment and updating the strings.