search

stanford-crfm / helm

Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in Holistic Evaluation of Text-to-Image Models (HEIM) (https://arxiv.org/abs/2311.04287).
https://crfm.stanford.edu/helm
Apache License 2.0
1.85k stars 243 forks source link

Cohere and AI21 model names are changing #1524

Closed yifanmai closed 1 month ago

yifanmai commented 1 year ago

https://docs.cohere.com/changelog/model-names-are-changing

We should migrate cached results to reflect the new names, and then rename in them in the codebase.

The old names will work for a while, so this is not urgent.

yifanmai commented 11 months ago

AI21 model names have also changed: https://docs.ai21.com/changelog/jurassic-2-model-renaming-no-action-needed-thanks-to-automatic-rerouting

yifanmai commented 1 month ago

The affected Cohere and AI21 models are deprecated or will be deprecated soon, so this is considered wontfix.