It just stops working after a while

[7cripter]

Bug description Everything works fine at startup, except that some content is sent with empty text, but it's because of commas, it seems. But my main problem is that after some time it stops working absolutely any text or works with 20-30 attempts, the error is:message='Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8', url=URL('https://cn.bing.com/turing/conversation/create?bundleVersion=1.1199.4').

I have tried a lot of things and I don't understand why this error occurs, I can send a message a billion times and it will most likely not be processed, but as soon as I restart the script the same message will be processed immediately with 1 time. The exact time after which it stops working is difficult to say, but according to my observations from 12 hours to 2 days, more than 2 days my bot on this technology can not work inprintsepi, from the amount of content it does not seem to depend, only time.

The essence of my script is that I run a server on Python from which I then access from the server on Node JS, but I do not think that the problem may be in this, but I also changed several servers and approaches on Python and the result is always the same. I do not know how to fix this error and would be very grateful for help.

Environment

• python version: 3.11.2
• location: Host in OVH Poland

My code

from aiohttp import web
async def handle_post(request):
    try:
        import g4f

        g4f.debug.logging = True  # enable logging
        g4f.check_version = False  # Disable automatic version checking

        print(f"Data received 1")
        data_received = await request.json()
        print(f"Data received 2")

        response_data = await g4f.ChatCompletion.create_async(
            model="gpt-4",
            provider=g4f.Provider.Bing,
            messages=data_received,
            timeout=260,
        )

        print(f"ChatCompletion 2: {response_data}")

        return web.json_response(response_data)

    except Exception as e:
        print(f"ERROR: {e}")
        return ""

app = web.Application()
app.router.add_post('/post_data', handle_post)

if __name__ == '__main__':
    web.run_app(app, host='localhost', port=6789)

[Luneye]

will try to identify and fix this issue as soon as I can!

[Luneye]

I just wanted to let you know that someone else had a similar problem, and it had something to do with their IP address. Bing has a detection bot that flags certain IP addresses, and this may be what you're facing. Perhaps if you try using a VPN, or a residential proxy if you have one, that might solve your problem.

[Luneye]

In this repo, they implemented a more complex system to create a conversation with Bing. This could fix your issue completely, but I am not sure tho. Try to implement something similar to this if you can, and tell me if it solved your problem!

https://github.com/waylaidwanderer/node-chatgpt-api/blob/142720f043c6515b542d04990674507c799a23bf/src/BingAIClient.js#L43

[7cripter]

Thanks for the reply. But do you think using a proxy can solve the problem? I'll clarify again: although my script stops working after a while, it will work immediately after restarting the python script, so it seems to exclude the possibility that my IP gets into some kind of blocking from Bing

[Luneye]

Okay, I understand now! I apologize for not thoroughly examining your entire issue, but I believe I have some recommendations for you. Here's how I revised your code:

In my opinion, you should import the 'g4f' module outside of your 'handle_post()' function. This way, it won't import the entire module with each call of your function. Additionally, I made sure to replace 'g4f.check_version' (which doesn't exist) with 'g4f.version_check,' as that's what you intended. I also removed the 'timeout' parameter, I wasn't sure if it was causing your problem, so I opted to remove it.

Finally, please ensure that you always send your data in the following format: data = [{"role": "user", "content": "hi"}], and on your next call, you can push the AI answer so that your array will look like this data = [{"role": "user", "content": "hi"}, {"role": "assistant", "content": ai_answer}]

Here is the corrected Python code, which should work. Please note that I added CORS settings for testing purposes, but you should adapt/remove them if you plan to use this code in a production server:

from aiohttp import web
import aiohttp_cors
import g4f

async def handle_post(request):
    try:

        g4f.debug.logging = True  # enable logging
        g4f.version_check = False  # Disable automatic version checking

        data_received = await request.json()
        print(f"Data received: {data_received}")
        response_data = await g4f.ChatCompletion.create_async(
            model=g4f.models.gpt_4,
            provider=g4f.Provider.Bing,
            messages=data_received,
        )

        print(f"ChatCompletion: {response_data}")

        return web.json_response(response_data)

    except Exception as e:
        print(f"ERROR: {e}")
        return ""

app = web.Application()
cors = aiohttp_cors.setup(app)
for route in list(app.router.routes()):
    cors.add(route, {
        "*": aiohttp_cors.ResourceOptions(
            allow_credentials=True,
            allow_headers="*",
            allow_methods="*",
            allow_origin="*",
        )
    })
app.router.add_post('/post_data', handle_post)

async def handle_options(request):
    response = web.Response()
    response.headers['Access-Control-Allow-Origin'] = '*'
    response.headers['Access-Control-Allow-Methods'] = '*'
    response.headers['Access-Control-Allow-Headers'] = '*'
    return response

# Add the OPTIONS route to handle preflight requests
app.router.add_route('OPTIONS', '/post_data', handle_options)

if __name__ == '__main__':
    web.run_app(app, host='localhost', port=6789)

Here is also how the request processing logic should look:

async function chat_request(data) {
  const url = "http://127.0.0.1:6789/post_data";

  try {
    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify(data),
    });

    if (!response.ok) {
      throw new Error("Network response was not ok");
    }

    const response_data = await response.json();
    console.log("Response:", response_data);
    return response_data;
  } catch (error) {
    console.error("An error happened:", error);
    throw error;
  }
}

// here is how you could chat with your AI

var chat = new Array()
chat.push({"role": "user", "content": "hi"})
var response = await chat_request(chat)
chat.push({"role": "assistant", "content": response})
chat.push({"role": "user", "content": "interesting, could you explain to me what is a sword?"})
response = await chat_request(chat)
chat.push({"role": "assistant", "content": response})
chat.push({"role": "user", "content": "wow, cool!"})
....

Hope that was clear enough!

[7cripter]

Thank you very much! I have already installed this code on the server and will test it and after a while I will reply if it fixed my problem.

About the fact that in my original code import g4f inside the function, I did it because before the import was at the beginning of the file and there was the same error, I thought maybe together with the import are called some additional updates for example sessions or something similar, but even so there was this error. About your example with requests, yes I have everything exactly the same, but for the initial prompt I use role: "system", but it should not interfere? Also timeout, I added it later after I started having this error, I don't think it can be the reason.

[Luneye]

You're welcome man! Thanks for replying to my comments as well :)

I get your idea, don't worry tho, there is no global session that could cause some weird issues. You can relax letting g4f being called at the top of your code. You're right, sending a system prompt should not interfere at all. If you want tho, you can send me your code with your requests to Bing, to make sure everything is alright! Regarding the timeout, I would suggest not using it because sometimes Bing can also be very slow to respond. There is already a default timeout, which is the same as the one in the official Bing AI web app code. But yeah, that doesn't seem to be the cause of your problem anyway.

[7cripter]

Hello again, I may be boring you by now but my problem is not solved. The code worked for about 18 hours and then without any adequate reason started to give the same error as before, this time I decided to restart only the python script and it helped as expected the problem occurs in this part, the requests I passed were also identical, I don't know what exactly can break there but I doubt it's on the bing side. If need be I can test the same by passing completely empty data with only 1 message from the user although I'm more than sure the result will be the same.

I also have an assumption that there might be a memory leak or something similar in some place, maybe it makes sense to check just endlessly forcing to generate a lot of content and send it, although this is just a guess.

[Luneye]

Well, I may have the answer to your problem! It might be because you're sending too many messages to Bing. Bing already has a limit of 30 messages per conversation. There is also a limit on the number of characters you can use in a single message. I'm not sure if the server allows you to bypass it, but there is definitely a token limit at some point that you can't bypass. That may be the reason why you're getting this error.

I've also checked the code for a potential memory leak, but it doesn't seem to be the case, you're just having a server-side error :)

[7cripter]

Of course there are limitations, I realize that. But the problem is that I send exactly the same messages when I get an error before reloading the script and after and before reloading I get a bing error and after reloading everything works perfectly. I guess I can also wait as long as I want after it stops working and it doesn't start working until I just reload the script.

I will later provide a better example of what I send and receive later when this error occurs again.

[7cripter]

Hello again. I have supplemented my script to make sure of the problem. I now run the python process directly from NodeJS and can reload it and also send a completely clean message array with only 1 message. I wanted this to give me the ability to output to the console a specific history of this error with a single message and immediately after reloading the python script a response to exactly the same prompt, this is basically how it worked, I sent a bunch of times one message and got the error, then I reloaded just the python script through my nodejs script and the same message was handled the first time, however because it is now running through child_process it does not print correctly from my python script, I don't know why this is the case but because of this I can't provide you with specific logs for this, just know that this is how it works. There are no other factors that could interfere in any way, everything was working fine before (about 8 hours ago). I really hope you can fix this problem otherwise I'll just have to reboot the python server all the time, I really like your project and would like it to be better and evolve. If you need more information I am always in touch.

[Luneye]

Hey! I will still do my best to try and figure out what's happening on your side :). If it's possible for you, you could send me the code of what you're running on your server so that I can test it on my end as well. If not, you can simply go into the Bing provider file and insert the following lines between lines 448 and 449:

print("\nmessage sent\n", create_message(conversation, prompt, tone, context))

This will print the formatted message for Bing. Next, add the following lines between lines 74 and 75:

content = await response.text()
print("\nserver response from trying to create a conversation\n", content)

Please try running the Python script separately to send me the logs so that I will be able to see what's happening. Keep your hopes up, I am sure we will solve this issue soon man!

[7cripter]

Thank you. My current python script is your code you gave me based on mine. I will later change to running the python server manually and add the code you gave me for more specific testing. I think on your end you could try just using the same code you provided me and use it without restarts for a few days, preferably sending a lot of content, although I'm not 100% sure it depends on the amount of content.

[Luneye]

Okay, I may have identified a potential issue in the code. This could be related to async/await calls, and it might be the reason why you encounter this problem when creating conversations multiple times. I am currently investigating this in-depth and will provide a fix as soon as possible!

[Luneye]

I believe I've fixed it! You can either wait until my pull request is accepted and update your g4f version, or you can fix the Bing provider code by yourself. To do so, just replace every instance of "async with await" that you find with just "async with." This should resolve the issue!

[7cripter]

Thank you very much! I'll wait until you update the repository and update the dependencies, then i tell if this problem happens again or not. I would also like to help fix the problem when certain messages return an empty string like "" or " ".

[7cripter]

Hello again, I updated the repository, I saw that you removed await. But it didn't help, after a while the same thing happens.

[Luneye]

Will try to look deeper through the creation of a conversation because it seems that many people are getting errors there.

[github-actions[bot]]

Bumping this issue because it has been open for 7 days with no activity. Closing automatically in 7 days unless it becomes active again.

[Dark-V]

Seems, same issue here:

1347

1305

1036

1136

1280

[Dark-V]

Looks like need closed, fixed in

1356

1364

[github-actions[bot]]

Bumping this issue because it has been open for 7 days with no activity. Closing automatically in 7 days unless it becomes active again.

[github-actions[bot]]

Closing due to inactivity.