Closed Max-Jesch closed 6 months ago
Hey @Max-Jesch,
what about an example like this?
from dotenv import load_dotenv
from genai.client import Client
from genai.credentials import Credentials
from genai.schema import (
TextGenerationParameters,
TextGenerationReturnOptions,
)
def heading(text: str) -> str:
"""Helper function for centering text."""
return "\n" + f" {text} ".center(80, "=") + "\n"
# make sure you have a .env file under genai root with
# GENAI_KEY=<your-genai-key>
# GENAI_API=<genai-api-endpoint>
load_dotenv()
client = Client(credentials=Credentials.from_env())
print(heading("Simple Text Generation"))
# yields batch of results that are produced asynchronously and in parallel
for response in client.text.generation.create(
model_id="google/flan-t5-xl",
inputs=["What is a molecule?", "What is NLP?"],
parameters=TextGenerationParameters(
max_new_tokens=150,
min_new_tokens=20,
return_options=TextGenerationReturnOptions(
input_text=True,
),
),
):
result = response.results[0]
print(f"Input Text: {result.input_text}")
print(f"Generated Text: {result.generated_text}")
print("")
@Tomas2D This looks really good. Thanks for the fast response :-) Is this from the documentation somewhere, or did I just miss this?
Anyway: It gives me an error, that I dont understand. Do you an idea about that?
============================ Simple Text Generation ============================
JSONDecodeError Traceback (most recent call last) Cell In[10], line 24 22 print(heading("Simple Text Generation")) 23 # yields batch of results that are produced asynchronously and in parallel ---> 24 for response in client.text.generation.create( 25 model_id="google/flan-t5-xl", 26 inputs=["What is a molecule?", "What is NLP?"], 27 parameters=TextGenerationParameters( 28 max_new_tokens=150, 29 min_new_tokens=20, 30 return_options=TextGenerationReturnOptions( 31 input_text=True, 32 ), 33 ), 34 ): 35 result = response.results[0] 36 print(f"Input Text: {result.input_text}")
File c:\Users\MaximilianJesch\Documents\GitHub\watsonx_for_social_media.venv\lib\site-packages\genai\text\generation\generation_service.py:210, in GenerationService.create(self, model_id, prompt_id, inputs, parameters, moderations, data, execution_options) 206 execution_options_formatted.callback(response) 208 return response --> 210 yield from execute_async( 211 inputs=prompts, 212 handler=handler, ... 354 except StopIteration as err: --> 355 raise JSONDecodeError("Expecting value", s, err.value) from None 356 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0) Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...
The example is in the documentation (yet). I'm not aware of this error. Please ensure you use the correct API and the latest SDK version (2.1.1).
What type of documentation needs to be updated? [X] API documentation [ ] Code of Conduct (CoC) [ ] Contribution Guide [ ] README file [ ] Project Team
In my opinion the documentation is missing a "hello world" example of how you send a prompt to a model and get a result back. The only thing I could find is an asynchronos implementation which sure is useful but is unnecessarily complex to get started https://ibm.github.io/ibm-generative-ai/v2.1.1/rst_source/examples.text.generation.html