UnicodeDecodeError: 'utf-8' codec can't decode bytes in position x-y: unexpected end of data

[eivl]

  File "C:\Users\eivin\AppData\Local\pypoetry\Cache\virtualenvs\mychat-BEEt1f8j-py3.12\Lib\site-packages\twitch\twitch.py", line 165, in read
    for recv in self.__recv():
  File "C:\Users\eivin\AppData\Local\pypoetry\Cache\virtualenvs\mychat-BEEt1f8j-py3.12\Lib\site-packages\twitch\twitch.py", line 151, in __recv
    recv = self.sock.recv(1024).decode("utf-8")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 434-435: unexpected end of data

I have seen this a few times today, not sure what causes it. It would need to be some sort of corrupted data?

Im open for a discussion of how I could troubleshoot it if anyone knows.

[eivl]

here is the total at the point where it fails.

'@badge-info=subscriber/29;badges=subscriber/24,premium/1;client-nonce=bdf6642353ca0093dd93682641171029;color=#FF7F50;display-name=Deluziaa;emotes=;first-msg=0;flags=;id=1d15d317-d232-43a2-bb26-f99fab0ab9b5;mod=0;returning-chatter=0;room-id=56649026;subscriber=1;tmi-sent-ts=1696786320360;turbo=0;user-id=142860814;user-type= :deluziaa!deluziaa@deluziaa.tmi.twitch.tv PRIVMSG #quin69 :X⣿⣿⣿⣿⡟⠛⠁⠄⠄⠄⠄⢀⣀⣀⠄⠄⠄⠄⣤⣽⣿⣿⣿⣿⣿⣿⣿⣿X X⣿⣿⣿⡋⠁⠄⠄⠄⣠⣶⣾⣿⣿⣿⣿⠄⢦⡄⠐⠬⠛⢿⣿⣿⣿⣿⣿⣿X X⣿⡿⠇⠁⠄⠄⣠⣾⣿⣿⡿⠟⠋⠁⠄⠄⠈⠁⠄⠄⠄⠄⠙⢿⣿⣿⣿⣿X X⣿⠃⠄⠄⠄⠘⣿⣿⣿⣿⢀⣠⠄⠄⠄⠄⣰⣶⣀⠄⠄⠄⠄⠸⣿⣿⣿⣿X X⣏⠄⠄⠄⠄⠄⣿⣿⣿⡿⢟⣁⠄⣀⣠⣴⣿⣿⠿⠷⠶⠒⠄⠄⢹⣿⣿⣿X X⡏⠄⠄⠄⠄⢰⣿⣿⣿⣿⣿⣿⣿⣿⡟⠄⠛⠁⠄⠄⠄⠄⠄⠄⢠⣿⣿⣿X X⡇⠄⠄⠄⠄⠈⢿⣿⣿⣿⣿⣿⣿⣿⡇⠄⣼⣿⠇⠘⠄⠁⠄⠄⠄⢻⣿⣿X X⣇⠄⠄⠄⠄⠄⠸⢿⣿⣿'

Im pretty sure there are some moderation tools kicking in because I cannot find this message in the normal twitch chat.

[eivl]

X⣿⣿⣿⣿⡟⠛⠁⠄⠄⠄⠄⢀⣀⣀⠄⠄⠄⠄⣤⣽⣿⣿⣿⣿⣿⣿⣿⣿X
X⣿⣿⣿⡋⠁⠄⠄⠄⣠⣶⣾⣿⣿⣿⣿⠄⢦⡄⠐⠬⠛⢿⣿⣿⣿⣿⣿⣿X 
X⣿⡿⠇⠁⠄⠄⣠⣾⣿⣿⡿⠟⠋⠁⠄⠄⠈⠁⠄⠄⠄⠄⠙⢿⣿⣿⣿⣿X 
X⣿⠃⠄⠄⠄⠘⣿⣿⣿⣿⢀⣠⠄⠄⠄⠄⣰⣶⣀⠄⠄⠄⠄⠸⣿⣿⣿⣿X 
X⣏⠄⠄⠄⠄⠄⣿⣿⣿⡿⢟⣁⠄⣀⣠⣴⣿⣿⠿⠷⠶⠒⠄⠄⢹⣿⣿⣿X 
X⡏⠄⠄⠄⠄⢰⣿⣿⣿⣿⣿⣿⣿⣿⡟⠄⠛⠁⠄⠄⠄⠄⠄⠄⢠⣿⣿⣿X 
X⡇⠄⠄⠄⠄⠈⢿⣿⣿⣿⣿⣿⣿⣿⡇⠄⣼⣿⠇⠘⠄⠁⠄⠄⠄⢻⣿⣿X 
X⣇⠄⠄⠄⠄⠄⠸⢿⣿⣿

Im guessing some mod tool deletes the message before it can be completely transmitted over the sockets.

[eivl]

At the moment, I just added an infinite asynchronous iterator to reconnect automatically on errors. It has been stable for a few minutes so it looks ok.

I also added this

    def __recv(self) -> Generator[str, None, None]:
        """
        Receive a message from the server.
        :return: A generator of messages.
        """
        total = ""
        while True:
+            try:
+               recv = self.sock.recv(1024).decode("utf-8")
+            except UnicodeDecodeError as e:
+               total = ""
+               continue
            total += recv
            if "\n" not in total:
                continue

            for line in total.split("\n"):
                yield line
                total = line

I broke indentation, please don't copy paste as is :D

[eloonstra]

Once again, thanks for contributing! I know what the issue and the next release will contain the fix.