Issue with Image Generation in HugChat API using Query Method

[karan7864]

Description: I am encountering an issue when attempting to generate an image using the HugChat API. After creating an instance of a new conversation with an assistant, I cannot generate an output with a query request. I am uncertain if I am using the query method correctly for this purpose. Should I be using the chat method or stream_query instead? Additionally, how can I display the image or save it somewhere? also the output along with query are not getting visible in conversation in hugging chat application

Code to Reproduce: `from hugchat import hugchat

Initialize chatbot with cookie path

chatbot = hugchat.ChatBot(cookie_path=COOKIE_PATH) current_model = "CohereForAI/c4ai-command-r-plus"

Specify assistant ID

ASSISTANT_ID = "6656d6b5f6b67c0476920e00"

Create a new conversation with the assistant

conversation = chatbot.new_conversation(assistant=ASSISTANT_ID) print(f"Current model is: {current_model}")

Attempt to generate an image

query_result = chatbot.query("generate image of a dog and cat!", web_search=True)

` Expected Behavior: The query method should generate an image based on the given prompt and either display it or provide a way to save it.

Actual Behavior: The image is not generated, and there is no clear indication of how to proceed with displaying or saving the image.

Questions:

Am I using the query method correctly for image generation? Should I be using the chat method or stream_query instead for this purpose? How can I display the generated image or save it to a file?

Thank you for your assistance! @Soulter

[github-actions[bot]]

Hi! Thanks for your issue, we will deal with your issue as soon as possible.

[Soulter]

Hello, I got the same problem, I will fix the bug soon

UPDATE: Your query method is correct. It seems like your assistant return a text like Generating your image...\r\n![](https://pollinations.ai/p/cat), hugchat cannot help you to save the image. You should implement that on your own.

[karan7864]

Dear @Soulter,

Thank you for your prompt response to my issue regarding image generation using the HugChat API. I appreciate the clarification that the query method is correctly implemented for this purpose.

However, I have encountered an additional problem. The query method is returning an empty string as its response. Here is an example of the output:

`

query_result.text '' `

As per your assistance , I intend to extract the URL from the response text, similar to the following example:

Generating your image...\r\n

Unfortunately, with the current response being an empty string, I am unable to proceed with fetching the image URL and making a GET request to the server.

Could you please help identify why the query method might be returning an empty string? Any insights or additional steps to ensure the correct response would be greatly appreciated.

for your reference this is all what my query_result looks like with printing all values of object attributes values:

I look forward to your feedback and any further assistance you can provide to resolve this issue.

Thank you for your support.

[Soulter]

Oh I found a bug and use query_result.text may not work You can use this method as a temporary method:

query_result = chatbot.query("generate image of a dog and cat!", web_search=True)
text_result = str(query_result) # this is the pure text that llm returned

I am fixing the bug

[Soulter]

And when you created a conversation, you can pass the parameter switch_to=True to activate the conversation, like this:

chatbot.new_conversation(assistant="6656d6b5f6b67c0476920e00", switch_to=True)

[karan7864]

ohh it resolved my issue thanks for your help Souther) between i am curious as i was just going though directories structure and its working i found generator object so i just tried to create url using it the code i used is as follow:

from hugchat import hugchat
import requests
from PIL import Image
from io import BytesIO

# Initialize chatbot with cookie path
chatbot = hugchat.ChatBot(cookie_path=COOKIE_PATH)
current_model = "CohereForAI/c4ai-command-r-plus"

# Specify assistant ID
ASSISTANT_ID = "6656d6b5f6b67c0476920e00"

# Create a new conversation with the assistant
conversation = chatbot.new_conversation(assistant=ASSISTANT_ID,switch_to=True)
print(f"Current model is: {current_model}")

# Attempt to generate an image
query_result = chatbot.query("generate image of car flying in air with a red car in the background!", web_search=True)

# Extract image details from the generator output
def extract_image_details(gen):
    image_url = None
    for item in gen:
        if item['type'] == 'file' and item['mime'] == 'image/webp':
            image_name = item['name']
            image_sha = item['sha']
            # Construct the URL (assuming a base URL, you may need to adjust this)
            base_url = "https://pollinations.ai/p/"
            image_url = f"{base_url}{image_sha}.webp"

            break
    return image_url

# Get the generator and extract image URL
gen = query_result.g
image_url = extract_image_details(gen)
print(image_url)
# Fetch and display the image if URL is found
if image_url:
    response = requests.get(image_url)
    image = Image.open(BytesIO(response.content))
    image.show()
else:
    print("Image URL not found in the generator output.")

It actually generates an image, but somehow it fetches the wrong image from the server or something similar. Can you let me know why this might be happening or if I am doing something wrong?

Thank you again for updating about that issue)

[Soulter]

Maybe you should use regex to get the url from result. It seems like the generator will not return an item that type is 'file' or 'mime'

[karan7864]

Actually, it is working, but I don't understand why it is fetching the wrong image from the server. Could you please run the snippet on your end? In my case, it returned the following URL, which is clearly not of a flying car:

https://image.pollinations.ai/prompt/eb8f9542a186c77ca31a6126cb3a73b88d2a72373bf54ade6df2d6c5c77c52c9.webp UPDATE: it doesn't work always as generator have some internal issue it returns empty sometimes when load on pollinantions ai is higher.......

NEW UPDATE:

from hugchat import hugchat
import requests
from PIL import Image
from io import BytesIO
import re

# Initialize chatbot with cookie path
chatbot = hugchat.ChatBot(cookie_path=COOKIE_PATH)
current_model = "CohereForAI/c4ai-command-r-plus"

# Specify assistant ID
ASSISTANT_ID = "6656d6b5f6b67c0476920e00"

# Create a new conversation with the assistant
conversation = chatbot.new_conversation(assistant=ASSISTANT_ID, switch_to=True)
print(f"Current model is: {current_model}")

# Attempt to generate an image
query_result = chatbot.query("generate image of car flying in air with a red car in the background!", web_search=True)

# Use regex to extract image URL from generator output
def extract_image_url_from_generator(generator):
    url_pattern = re.compile(r'https://\S+')
    final_text = ""

    for item in generator:
        item_str = str(item)  # Convert the generator item to a string
        print(f"Generator item: {item_str}")  # Debug: print generator item
        if item['type'] == 'finalAnswer':
            final_text = item['text']
            break

    match = url_pattern.search(final_text)
    if match:
        return match.group(0)
    return None

# Extract image URL using regex from the generator object
try:
    gen = query_result.g
    image_url = extract_image_url_from_generator(gen)
    print(f"Extracted image URL: {image_url}")
except Exception as e:
    print(f"Failed to extract image URL: {e}")

# Fetch and display the image if URL is found
if image_url:
    try:
        response = requests.get(image_url)
        response.raise_for_status()  # Raise an HTTPError for bad responses (4xx and 5xx)
        image = Image.open(BytesIO(response.content))
        image.show()
    except requests.exceptions.RequestException as e:
        print(f"HTTP error occurred: {e}")
    except Image.UnidentifiedImageError:
        print("The URL did not point to a valid image.")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")
else:
    print("Image URL not found in the generator output.")

thats working :) for me but not so usefull as Soulter already made changes in API thanks for assistance)

[Soulter]

You can update to the latest version 0.4.8 to use image generation capability:

# Create your ChatBot
chatbot = hugchat.ChatBot(cookies=cookies.get_dict())  # or cookie_path="usercookies/<email>.json"

message_result = chatbot.chat("Hi!") # note: message_result is a generator, the method will return immediately.

# Non stream
message_str: str = message_result.wait_until_done() # you can also print(message_result) directly. 
# get files(such as images)
file_list = message_result.get_files_created() # must call wait_until_done() first!

# tips: model "CohereForAI/c4ai-command-r-plus" can generate images :)