Inputs doesn’t work for BAM REST API V2

[hzpeng]

Version Information

Package Version: ibm-generative-ai 2.1.0 Operating System: MacOS

What is the expected behavior?

Generate text

What is the actual behavior?

Causes an exception:

genai.exceptions.ApiResponseException: Failed to handle request to https://bam-api.res.ibm.com/v2/text/generation?version=2023-11-22.
{
  “error”: “Not Found”,
  “extensions”: {
    “code”: “NOT_FOUND”,
    “state”: null
  },
  “message”: “Route does not exist”,
  “status_code”: 404
}

Code(original sample in migration guide to V2 for python)

from dotenv import load_dotenv

from genai.client import Client
from genai.credentials import Credentials
from genai.schema import (
    TextGenerationParameters,
    TextGenerationReturnOptions,
)

def heading(text: str) -> str:
    """Helper function for centering text."""
    return "\n" + f" {text} ".center(80, "=") + "\n"
# make sure you have a .env file under genai root with
# GENAI_KEY=<your-genai-key>
# GENAI_API=<genai-api-endpoint>
load_dotenv()
client = Client(credentials=Credentials.from_env())

print(heading("Simple Text Generation"))
# yields batch of results that are produced asynchronously and in parallel
for response in client.text.generation.create(
    model_id="google/flan-t5-xl",
    inputs=["What is a molecule?", "What is NLP?"],
    parameters=TextGenerationParameters(
        max_new_tokens=150,
        min_new_tokens=20,
        return_options=TextGenerationReturnOptions(
            input_text=True,
        ),
    ),
):
    result = response.results[0]
    print(f"Input Text: {result.input_text}")
    print(f"Generated Text: {result.generated_text}")
    print("")

Comment the inputs:

from dotenv import load_dotenv

from genai.client import Client
from genai.credentials import Credentials
from genai.schema import (
    TextGenerationParameters,
    TextGenerationReturnOptions,
)
def heading(text: str) -> str:
    """Helper function for centering text."""
    return "\n" + f" {text} ".center(80, "=") + "\n"

# make sure you have a .env file under genai root with
#  GENAI_KEY=<your-genai-key>
#  GENAI_API=<genai-api-endpoint>
load_dotenv()
client = Client(credentials=Credentials.from_env())

print(heading("Simple Text Generation"))
# yields batch of results that are produced asynchronously and in parallel
for response in client.text.generation.create(
    model_id="google/flan-t5-xl",
    # inputs=["What is a molecule?", "What is NLP?"],
    parameters=TextGenerationParameters(
        max_new_tokens=150,
        min_new_tokens=20,
        return_options=TextGenerationReturnOptions(
            input_text=True,
        ),
    ),
):
    result = response.results[0]
    print(f"Input Text: {result.input_text}")
    print(f"Generated Text: {result.generated_text}")
    print("")

The result is as following:

============================ Simple Text Generation ============================  

The exception disappears, because input is none, then the generated text is also none. I also called BAM REST API V2 through curl command, the curl call is successful as following.

curl https://bam-api.res.ibm.com/v2/text/generation?version=2023-11-22 \
-X POST \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer pak-XXXXXXXX' \
-d '{
  "model_id": "google/flan-t5-xxl",
  "input": "Hello world",
  "parameters": {
    "max_new_tokens": 200
  },
  "moderations": {
    "hap": {
      "input": true,
      "output": true,
      "threshold": 0.1
    }
  }
}'

By using curl command, we must use input and in the "Migration guide to v2", it mentions:

Text generation (v2/text/generation)
'inputs' is changing to 'input', accepting only single string

While for IBM-Generative-AI Python SDK, we must use inputs and the call will fail. More details, please reference above exception. Could you please check and give a sample about how to call BAM REST API V2 using IBM-Generative-AI Python SDK? Thanks!

[Tomas2D]

The example provided under the title Code(original sample in migration guide to V2 for Python) works correctly. When using curl, you must provide an input, whereas for the SDK, you must provide inputs.

[hzpeng]

Thanks for your quick reply. The example provided under the title Code(original sample in migration guide to V2 for Python) will always cause the following exception. Could you pls help on this ? Thanks.

genai.exceptions.ApiResponseException: Failed to handle request to https://bam-api.res.ibm.com/v2/text/generation?version=2023-11-22.
{
  “error”: “Not Found”,
  “extensions”: {
    “code”: “NOT_FOUND”,
    “state”: null
  },
  “message”: “Route does not exist”,
  “status_code”: 404
}

[Tomas2D]

Ensure your GENAI_API is set to https://bam-api.res.ibm.com.