Fix Subscription RemoteProtocolError

[User00092]

I encounter "httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)" when the server does not receive any updates within a specific time frame. While it is possible to work around this issue in certain cases by creating new data on the server that is ignored by the client, such an approach is not considered appropriate or ideal.

[m29h]

I can (somewhat) confirm this bug. For me when pocketbase closes the connection (after 300 seconds without events as per documentation) the EventLoop just ends (without an exception).

The effect is no more messages are received as the client currently does not realize that the event loop was forcibly closed and also does not try to reconnect to the realtime endpoint.

[m29h]

@User00092 can you have a look at the PR to see if that solves your original issue?

[User00092]

@m29h Unfortunately, that does not solve the issue. The exception that httpx raises results in self.kill being set to true. Changing self.kill = True to print('httpx closed connection.'), httpx closed connection. was displayed, and I was still able to receive updates from the server. I don't quite understand how you unsubscribe, so I cannot implement <Object>.kill = True for you. However, if possible, please do.

Here are the changes I made in pocketbase/services/utils/sse.py: line 8:

import time

line 90-100:

def run(self):
        while not self.kill:
            try:
                for event in self._events():
                    if self.kill:
                        break
                    if event.event in self.listeners:
                        self.listeners[event.event](event)
                time.sleep(0.5) #  Remove some stress off the CPU
            except Exception:
                pass

[m29h]

@User00092 can you give the output of pip show httpx so that I can try to reproduce this with the same version. It seems that the exception is not triggered in all httpx library versions...

[User00092]

@m29h of course, here is the information:

Name: httpx
Version: 0.23.3
Summary: The next generation HTTP client.
Home-page:
Author:
Author-email: Tom Christie <tom@tomchristie.com>
License:
Location: ---\venv\Lib\site-packages
Requires: certifi, httpcore, rfc3986, sniffio
Required-by: pocketbase

[User00092]

I edited the run function to better handle errors, along with the RemoteProtocolError if raised. This allows other exceptions to be handled properly, while handling RemoteProtocolError in a way that doesn't affect the program.

def run(self):
        while not self.kill:
            try:
                for event in self._events():
                    if self.kill:
                        break
                    if event.event in self.listeners:
                        self.listeners[event.event](event)
                time.sleep(1) #  Remove some stress off the CPU (In the event there are multiple threads), also saving system memory.
            except httpx.RemoteProtocolError:
                pass

            except Exception as e:
                print(e)  # Or raise it
                self.kill = True

[m29h]

Name: httpx
Version: 0.23.3

@User00092 just tried it again with the same version in my environment linux/python 3.11.3 explicitly installing httpx 0.23.3 and still do not get an exception. The httpx documentation says remote protocol error means "The protocol was violated by the server" this is somewhat not intuitively what one would expect if the other side silently closes the connection.

Is your pocketbase instance maybe behind some reverse proxy or firewall that does something strange? I digged a bit deeper and it turns out httpx is nowhere raising a RemoteProtocolError but only wrapping an exception that is incoming from the httpcore library. the httpcore library itself wraps the RemoteProtocolError exception incoming from the h11 library.

Looking at the h11 repository i found this (closed) issue: python-hyper/h11#133 from Year 2021. What version is your h11 library? If it is < 0.14.0 than maybe it helps to upgrade h11 to >=0.14.0