Closed alexandrasouly closed 1 month ago
The type annotation for the key is incorrect. It should be a string instead of an int.
responses = model.predict("hi?",logit_bias={'10':-13.})
The type annotations were fixed in 1.60: https://github.com/googleapis/python-aiplatform/releases/tag/v1.60.0
I'm running into the same issue as https://stackoverflow.com/questions/77805968/how-to-use-logit-bias-parameter-of-model-predict-in-vertexai-python-sdk when trying to use the logit_bias parameter in the Palm2 textegeneration API as documented in https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.language_models.TextGenerationModel#vertexai_language_models_TextGenerationModel_predict
Steps to reproduce
The following code using the documented logit_bias parameter results in an errror:
Stack trace