Unable to generate text with some tuned generative models

[adesso-dominik-chodounsky]

Description of the bug:

When tuning one of the generative models in google.generativeai.list_models(), you are unable to directly call some of the supported generation methods (or at least it is not quite clear how to call them).

For example, if we take the models/text-bison-001 model and train it using genai.create_tuned_model(), let the operation finish and load the model with genai.GenerativeModel(model_name=result.name) as suggested by all documentation, we are only able to call the method for generate_content which is however not supported by this model type.

Actual vs expected behavior:

I would expect the GenerativeModel class to support calling the supposedly supported methods of each of the possible generative models.

i.e. for the text-bison model, something like: GenerativeModel(model_name="models/text-bison").generate_text("Hello world").

This currently ends with: AttributeError: 'GenerativeModel' object has no attribute 'generate_text'

The method that is implemented for generating content results in the following:

GenerativeModel(model_name="models/text-bison").generate_content("Hello world")

NotFound: 404 models/text-bison is not found for API version v1beta, or is not supported for generateContent. Call ListModels to see the list of available models and their supported methods.

Any other information you'd like to share?

No response

[Gunand3043]

@adesso-dominik-chodounsky

Just confirming: Are you using the Vertex AI SDK or Google AI SDK to tune the model?

[adesso-dominik-chodounsky]

@Gunand3043 I am using the Google AI SDK (import google.generativeai as genai).

[Gunand3043]

@adesso-dominik-chodounsky

Currently, Google AI SDK support gemini-1.0-pro-001 and gemini-1.5-flash-001-tuning for tuning. You can use the following code to check the available models for tuning:

for m in genai.list_models():
    if "createTunedModel" in m.supported_generation_methods:
        print(m.name)

Once you have tuned the model, you can call it by replacing the model name with the name used during tuning, for example: model = genai.GenerativeModel(model_name=f'tunedModels/{name}')

Below code is to check list of models that support generateContent:

for m in genai.list_models():
    if "generateContent" in m.supported_generation_methods:
        print(m.name)

Use the generate_content method to get the response.

You can find the detailed tuning notebook here.

[adesso-dominik-chodounsky]

@Gunand3043 I see, the tuning code seemed to work for other models up until the point of content generation with the tuned model, so I did not explore the base compatibility further, thank you for the correction.