search

GiusTex / EdgeGPT

Extension for Text Generation Webui based on EdgeGPT, a reverse engineered API of Microsoft's Bing Chat AI
125 stars 5 forks source link

asyncio.exceptions.TimeoutError in Colab #25

Closed mrdc closed 1 year ago

mrdc commented 1 year ago

Hello,

When running Hey Bing, I get asyncio.exceptions.TimeoutError in EdgeGPT() func, which awaits bot.ask(). Are there some sort of hardcoded timeout?

P.S. script.py was edited to fix the conda issue in Colab, so line numbers will differ, compared to original script.py ;)

Log:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/client.py", line 662, in __await_impl__
    await protocol.handshake(
  File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/client.py", line 329, in handshake
    raise InvalidStatusCode(status_code, response_headers)
websockets.exceptions.InvalidStatusCode: server rejected WebSocket connection: HTTP 200

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/client.py", line 655, in __await_impl_timeout__
    return await self.__await_impl__()
  File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/client.py", line 676, in __await_impl__
    await protocol.wait_closed()
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 427, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1323, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1067, in call_function
    prediction = await utils.async_iteration(iterator)
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 336, in async_iteration
    return await iterator.__anext__()
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 329, in __anext__
    return await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 312, in run_sync_iterator_async
    return next(iterator)
  File "/content/text-generation-webui/modules/chat.py", line 305, in generate_chat_reply_wrapper
    for i, history in enumerate(generate_chat_reply(text, state, regenerate, _continue, loading_message=True)):
  File "/content/text-generation-webui/modules/chat.py", line 290, in generate_chat_reply
    for history in chatbot_wrapper(text, state, regenerate=regenerate, _continue=_continue, loading_message=loading_message):
  File "/content/text-generation-webui/modules/chat.py", line 206, in chatbot_wrapper
    prompt = apply_extensions('custom_generate_chat_prompt', text, state, **kwargs)
  File "/content/text-generation-webui/modules/extensions.py", line 208, in apply_extensions
    return EXTENSION_MAP[typ](*args, **kwargs)
  File "/content/text-generation-webui/modules/extensions.py", line 86, in _apply_custom_generate_chat_prompt
    return extension.custom_generate_chat_prompt(text, state, **kwargs)
  File "/content/text-generation-webui/extensions/EdgeGPT/script.py", line 222, in custom_generate_chat_prompt
    asyncio.run(EdgeGPT())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/content/text-generation-webui/extensions/EdgeGPT/script.py", line 196, in EdgeGPT
    response = await bot.ask(prompt=UserInput, conversation_style=style, simplify_response=True)
  File "/usr/local/lib/python3.10/dist-packages/EdgeGPT/EdgeGPT.py", line 116, in ask
    async for final, response in self.chat_hub.ask_stream(
  File "/usr/local/lib/python3.10/dist-packages/EdgeGPT/chathub.py", line 102, in ask_stream
    async with connect(
  File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/client.py", line 637, in __aenter__
    return await self
  File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/client.py", line 654, in __await_impl_timeout__
    async with asyncio_timeout(self.open_timeout):
  File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/async_timeout.py", line 169, in __aexit__
    self._do_exit(exc_type)
  File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/async_timeout.py", line 252, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError
GiusTex commented 1 year ago

There is no harcoded timeout, it's the default asyncio function, that waits (await) the end of the edgegpt function. Colab may have some problems with asyncio functions, I'll check it tomorrow

mrdc commented 1 year ago

There is approx 200ms timeout in Colab, so nest_asyncio should be used, but it's not solving the issue, when I add nest_asyncio.apply() to the script.py. Have also tried a fix from this issue without success: https://github.com/googlecolab/colabtools/issues/1648

GiusTex commented 1 year ago

At first I used too nest_asyncio for colab and it did work, then I removed it, long story; can I use your fix to repair the colab notebook, if it works and you don't mind?

mrdc commented 1 year ago

Using nest_asyncio.apply() is not fixing the issue. I'm not deep in debugging yet - have made a brief look and trying easy fixes)

GiusTex commented 1 year ago

Ok, I'll check tomorrow what to do then

GiusTex commented 1 year ago

I can't work on the extension because Colab keep saying me: Authentication failed. You have not been accepted into the beta. both with and without cookies (I even tried naming them bing_cookies_*.json, as written here: readme. If you want I was making tests here: Copy of Text-generation-webui-EdgeGPT.ipynb.txt (remove .txt). Here EdgeGPT on Colab starts, but gives that error. If for you works I added a way to download models from a link, this way you can download a small model and make tests with it in the webui

mrdc commented 1 year ago

Authentication failed

Looks like Colab IPs are blocked by Microsoft. EdgeGPT works fine from my laptop even without log in, but fails in Colab(

GiusTex commented 1 year ago

Thanks for the info; the first time I released the colab notebook it worked, this must be a recent Microsoft thing, damn. I think I'll have to remove the notebook then...

mrdc commented 1 year ago

this must be a recent Microsoft thing

I've updated cookies and it looks like the auth issue is gone, but again the timeout issue is here:


InvalidStatusCode                         Traceback (most recent call last)

[/usr/local/lib/python3.10/dist-packages/websockets/legacy/client.py](https://localhost:8080/#) in __await_impl__(self)
    661             try:
--> 662                 await protocol.handshake(
    663                     self._wsuri,

10 frames

InvalidStatusCode: server rejected WebSocket connection: HTTP 200

During handling of the above exception, another exception occurred:

CancelledError                            Traceback (most recent call last)

CancelledError: 

During handling of the above exception, another exception occurred:

TimeoutError                              Traceback (most recent call last)

[/usr/local/lib/python3.10/dist-packages/websockets/legacy/async_timeout.py](https://localhost:8080/#) in _do_exit(self, exc_type)
    250         if exc_type is asyncio.CancelledError and self._state == _State.TIMEOUT:
    251             self._timeout_handler = None
--> 252             raise asyncio.TimeoutError
    253         # timeout has not expired
    254         self._state = _State.EXIT

TimeoutError: