SOLVED?: get_unread() stuck for me

[fire17]

Hi there! I'll start by sharing that i used to work with mukulhase/WebWhatsapp-Wrapper which was great but it is not maintained and you guys seem the most serious currently :P (many thanks) (I've got my license and everything)

So far managed to load wa-web and read messages from chats. seems good. But cant get new messages with the driver.get_unread() - it gets stuck and then spits out a timeout error.

more info: so basically i've cloned this repo (not pip) into folder named openwapy (easier to play with the code) and in my code I do (which works)

from openwapy.src import WhatsAPIDriver as waDriver
#...setup code
self.driver = waDriver(profile = profileDir, client='chrome', chrome_options=chrome_options, ............
#then
for contact in self.driver.get_unread():
#    ............

but the get_unread is just stuck. i've also tried to do execute with the function header (used to work sometimes with WebWhatsapp-Wrapper) but this just gets me None as output

Error from terminal

        ===================================
        Processing Incoming Messages
        ===================================

 ::: status is LoggedIn ::: 
XXX_STARTING_GET_UNREAD

Exception in thread Thread-19:
Traceback (most recent call last):
  File "/home/magic/wholesomegarden/WhatsappMaster/openwapy/src/wapi_js_wrapper.py", line 174, in __call__
    return self.driver.execute_async_script(command)
  File "/home/magic/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 659, in execute_async_script
    'args': converted_args})['value']
  File "/home/magic/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/magic/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: script timeout
  (Session info: chrome=89.0.4389.90)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "app.py", line 1208, in ProcessIncoming
    for contact in self.driver.get_unread():
  File "/home/magic/wholesomegarden/WhatsappMaster/openwapy/src/__init__.py", line 868, in get_unread
    raw_message_groups = self.wapi_functions.getUnreadMessages(include_me, include_notifications, True)
  File "/home/magic/wholesomegarden/WhatsappMaster/openwapy/src/wapi_js_wrapper.py", line 187, in __call__
    raise JsException("Error in function {0} ({1}). Command: {2}".format(self.function_name, e.msg, command))
openwapy.src.wapi_js_wrapper.JsException: Error in function getUnreadMessages (script timeout
  (Session info: chrome=89.0.4389.90)). Command: return WAPI.pyFunc(()=>WAPI.getUnreadMessages(true,false,true), arguments[0])

also, i've tried debugging what happens in the js code, but i cant seem to even see console.log("...") in the chrome console any ideas ?

I've been trying on my own for the last two days.. Please help lol THX a lot and have a good one! :grapes:

[fire17]

My current workaround: using getAllUnreadMessages and hashing message ids in self.history , and only return new messages, unseen in self.histoy. getUnreadMessages (the original method used in get_unread - without the "All") gets stuck, or returns None everytime. so this kinda works, but is not formatted correctly, and is makeing the response time worst..

current code works up to return self.factorUnread(new_messages)

def get_unread(self, include_me=False, include_notifications=False, use_unread_count=False):

        # Original & previous attempts
        # raw_message_groups = self.wapi_functions.getUnreadMessages(include_me, include_notifications, True)
        # code = "return WAPI.getUnreadMessages(false, false, false);"
        # raw_message_groups = self.driver.execute_script(script = code)

        # WorkAround
        code = "return WAPI.getAllUnreadMessages()"
        raw_message_groups = self.driver.execute_script(script = code)

        new_messages = []
        initHistory = False
        if len(self.history) == 0:
            initHistory = True # to skip the first load of however many unread messages you started with when loading

        for m in raw_message_groups:
            if m["id"] not in self.history: # checks if message already been through this workaround
                self.history.append(m["id"])
                if not initHistory:
                    if m["sender"] is not None:
                        print(m)
                        new_messages.append(m)

        for m in new_messages:
            print("NEEEWWWWWWWWWWWWWW MSG")
            print(m["sender"]["id"],"::::",m["content"])
            print()

        # return new_messages # **if do return here I get format error down the line, since its a dict and not a WhatsappObject
        if new_messages is not None:
            try:
                return self.factorUnread(new_messages) ######################### THROWS EXCEPTION
            except :
                traceback.print_exc()
                return []
        return []

    def factorUnread(self, raw_message_groups): #### THIS IS THE ORIGINAL FUNCTION THAT FORMATTED THE DATA COMMING FROM self.wapi_functions.getUnreadMessages
        unread_messages = []
        if raw_message_groups is not None:
            for raw_message_group in raw_message_groups:
                chat = factory_chat(raw_message_group, self)
                messages = list(
                    filter(None.__ne__, [factory_message(message, self) for message in raw_message_group['messages']]))
                messages.sort(key=lambda message: message.timestamp)
                unread_messages.append(MessageGroup(chat, messages))
                return unread_messages
        return []

this gives me all the message data but not really formatted correctly down the line

Error in factorUnread() trying to factor the unread raw_message_groups just like how WAPI.getUnreadMessages does

Traceback (most recent call last):
  File "/home/magic/wholesomegarden/WhatsappMaster/openwapy/src/__init__.py", line 889, in get_unread
    return self.factorUnread(new_messages)
  File "/home/magic/wholesomegarden/WhatsappMaster/openwapy/src/__init__.py", line 899, in factorUnread
    chat = factory_chat(raw_message_group, self)
  File "/home/magic/wholesomegarden/WhatsappMaster/openwapy/src/objects/chat.py", line 8, in factory_chat
    if js_obj["kind"] not in ["chat", "group", "broadcast"]:
KeyError: 'kind'

I dont really get how to patch this format, for now will probably extract important keys from it but I'll love to put everything back to normal. please help - i would love if someone could help me avoid this workaround and just use self.driver.get_unread() like normal :banana:

[mrodal]

Hi! could you try using driver.get_unread(use_unread_count=True) ?

[fire17]

first of all how do reset the counter for a group ? i need only the most recent unseen messages i'm asking cause it'll be good to know,

but .get_unread still gets stuck at self.wapi_functions.getUnreadMessages(include_me, include_notifications, use_unread_count) no matter what i tired

[fire17]

this is the error after it gets stuck

        ===================================
            Processing Incoming Messages
        ===================================

 ::: status is LoggedIn ::: 

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/openwa-1.3.15-py3.7.egg/openwa/wapi_js_wrapper.py", line 173, in __call__
    return self.driver.execute_async_script(command)
  File "/home/magic/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 659, in execute_async_script
    'args': converted_args})['value']
  File "/home/magic/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/magic/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out after 60000 ms

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "app.py", line 1280, in ProcessIncoming
    for contact in self.driver.get_unread():
  File "/usr/local/lib/python3.7/dist-packages/openwa-1.3.15-py3.7.egg/openwa/__init__.py", line 578, in get_unread
    raw_message_groups = self.wapi_functions.getUnreadMessages(include_me, include_notifications, use_unread_count)
  File "/usr/local/lib/python3.7/dist-packages/openwa-1.3.15-py3.7.egg/openwa/wapi_js_wrapper.py", line 186, in __call__
    raise JsException("Error in function {0} ({1}). Command: {2}".format(self.function_name, e.msg, command))
openwa.wapi_js_wrapper.JsException: Error in function getUnreadMessages (Timed out after 60000 ms). Command: return WAPI.pyFunc(()=>WAPI.getUnreadMessages(false,false,false), arguments[0])

[fire17]

Please help! what could be causing this? get_unread is not working on chrome nor firefox nor is it working when i change the source to pip version (ran pip install --upgrade openwa) and added

LocalWapi = False
if LocalWapi:
    from openwapy.src import WhatsAPIDriver as waDriver
else:
    from openwa import WhatsAPIDriver as waDriver

still gets stuck at get_unread

[darkash]

how many chatroom do you have? and could you try increase the driver timeout, increase it to like 10 mins and see if it still timed out

should not use that long timeout in production, but just curious whether it's the selenium or the javascript hanging

although rarely I had encountered the same issue on send_chat_message but it can be solved by retrying to send it again or completely ignoring it as the message actually get sent. Never used get_unread though

[fire17]

THHHANK YOU!!! @darkash indeed the issue was too many chats (and too many unseen messages) the first time it takes it a bit over a minute to load (77 seconds) , then works just fine, fast and all :)

the fix was as you suggested to raise the scripttimeout in init _ now the original functions works correctly :D

closing :smile:

[darkash]

THHHANK YOU!!! @darkash indeed the issue was too many chats (and too many unseen messages) the first time it takes it a bit over a minute to load (77 seconds) , then works just fine, fast and all :)

the fix was as you suggested to raise the scripttimeout in init _ now the original functions works correctly :D

closing 😄

I'm not suggesting to increase the script timeout permanently :p It's better to increase it initially if you've tons of unread messages then restart it with the default timeout value

Reason is if another function like chat_send_message has a problem with the node.js side (accidental bug or facebook changes) and its promise does not resolve, your python app won't wait for 10 mins like what I specified in reply. Unless your app won't really mind for that, I think the timeout should be kept as minimum as possible.

glad those actually helped tho

[fire17]

I just rasied it from 60 to 80 and it managed to go through and finish the job then in init in .get_unread() i looped through all chats sending the "seen" signal

_in heinsight maybe .get_all_chats() + .chat_send_seen() could have worked before .getunread() so sending "seen" could have happend without changing the timeout :thinking:

if markAllSeen or True:
    all_chats = self.get_all_chats()
    for m in all_chats:
        try:
            print("SEEN", m.id)
            self.chat_send_seen(m.id)
        except :
            traceback.print_exc()

deleted some unused groups and after all of that the time was much quicker (we're talking about the first time calling .get_unread() ), after the first time it works perfectly and reads new messages instantly

if someone is having these issues i suggest sending "seen" to all chats and cleaning up your chats if they're unused :) Thanks @darkash @mrodal ! :heart:

[fire17]

PLEASE REOPEN THIS ISSUE

I Just restarted my server, using the same code and again getting stuck at get_unread

The function finishes after 67 seconds so raising the timeout a bit is still relevant, BUT it used to take 60-something seconds the first time, THEN Runs Fast and Smoothly

now it takes around that time - EVERY ITERATION really dont want to use the prev workaround as it would skip messages sent one after another..

Please Help, this is really halting my workflow Thx

[darkash]

Can you try this script?

chats = driver.wapi_functions.getAllChatsWithNewMsg()
if chats and len(chats) > 0:
    for chat in chats:
        chat_id = chat['id']
        messages = driver.get_unread_messages_in_chat(chat_id)
        for msg in messages:
            print(message.content) # change this to your message processing logic

It's not a silver bullet, but the TimeoutError should occur less. I think it should work like get_unread by dividing the fetch message calls (slower though).

[fire17]

i dont know what is affecting this, but when it works well the first time get_unread takes sometime, then its working normally

my current code to get the messages:

unread = []
raw_message_groups = driver.get_unread()
for uu in raw_message_groups:
    chat = uu.chat
    for u in uu.messages:
        message = u.get_js_obj()
        unread.append(message)
    for msg in unread:
        print("NEEEWWWWWWWWWWWWWW MSG")
        print(msg["sender"]["id"],"::::",msg["content"])
        incomingCallback(msg)