replicate / replicate-python

Python client for Replicate
https://replicate.com
Apache License 2.0
771 stars 222 forks source link

What is the correct way to get full output text? #381

Closed paulocoutinhox closed 1 month ago

paulocoutinhox commented 1 month ago

Hi,

What is the correct way to get full output text?

In replicate dashboard i have the full text:

"A man with a fiery sword at his side, his face exposed, gazing directly into the camera, set against a mist-shrouded, medieval-style castle backdrop at dusk, in a stylized, cinematic realistic art style, featuring warm golden and crimson hues, with deep shadows and soft, ethereal lighting that casts an otherworldly glow, conveying a sense of heroism and mysticism, as if he is ready to charge into battle."

But when i get from python, i can only get one part of this:

exposed, gazing directly into the camera, set against a mist-shrouded, medieval-style castle backdrop at dusk, in a stylized, cinematic realistic art style, featuring warm golden and crimson hues, with deep shadows and soft, ethereal lighting that casts an otherworldly glow, conveying a sense of heroism and mysticism, as if he is ready to charge into battle."

Code example:

prompt_input = "give me a hello world in python"

new_prompt = replicate.run(
    "justmalhar/meta-llama-3.2-3b:5c11ba722650525b0ab1a877d1fde44c5dab7445072e9e8e1a6a47ec2026a4d4",
    input={
        "top_k": 50,
        "top_p": 0.9,
        "prompt": prompt_input,
        "max_tokens": 1024,
        "min_tokens": 0,
        "temperature": 0.6,
        "presence_penalty": 0,
        "frequency_penalty": 0,
    },
)

# get output
output = ""
for item in new_prompt:
    output += str(item)

print(output)
nickstenning commented 1 month ago

Hmmm. Something is definitely broken here. I'm looking into it.

nickstenning commented 1 month ago

We've got a fix for this in the works over in #383.

aron commented 1 month ago

Hi @paulocoutinhox we've just shipped 1.0.3 of the Python client which should address this issue. Please re-open this ticket if you are still running into issues.

paulocoutinhox commented 4 weeks ago

Thanks.