[evals] support `model` and soft-deprecate `model_name`

[dosubot[bot]]

🤖

Hey @mikeldking, great to see you back! Hope you're doing well.

Sure, it's possible to add support for model in the Arize Phoenix repository and softly deprecate model_name. This can be achieved by adding a new model attribute to the VertexAIModel class and modifying the _instantiate_model method to use this new attribute. If model is not provided, the method can fall back to using model_name. Here's how you can do it:

@dataclass
class VertexAIModel(BaseEvalModel):
    ...
    model: Optional[str] = None
    model_name: Optional[str] = None
    ...

    def _instantiate_model(self) -> None:
        if self.is_codey_model:
            from vertexai.preview.language_models import CodeGenerationModel  # type:ignore

            model = CodeGenerationModel
        else:
            from vertexai.preview.language_models import TextGenerationModel

            model = TextGenerationModel

        if self.tuned_model_name:
            self._model = model.get_tuned_model(self.tuned_model_name)
        elif self.model:
            self._model = model.from_pretrained(self.model)
        elif self.model_name:
            self._model = model.from_pretrained(self.model_name)
        else:
            raise ValueError("Either 'model' or 'model_name' must be provided.")

In this code, we first check if tuned_model_name is provided. If it is, we use it to get the tuned model. If it's not, we check if model is provided. If it is, we use it to get the pretrained model. If model is also not provided, we fall back to using model_name. If neither model nor model_name is provided, we raise a ValueError.

Please note that there are dependencies and functions in the Arize Phoenix codebase that rely specifically on the model_name attribute. For example, the model_name attribute is used in the _instantiate_model method and the is_codey_model property. If tuned_model_name is not provided, model_name is used to instantiate a pretrained model and to determine if the model is a Codey model. Therefore, softly deprecating model_name might require changes in these dependencies and functions as well.

You can find the relevant code in the src/phoenix/experimental/evals/models/vertexai.py file in the Arize Phoenix repository.

Sources

• src/phoenix/experimental/evals/models/vertexai.py

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.