Getting `StreamClosedError(f'Got {frame}')` when using the same RSD tunnel simulataneously

[morellexf13]

I guess my logic for iOS >= 17 should be something like this then:

try:
    host = 'fd30:9355:b54e::1'
    port = 60458
    with RemoteServiceDiscoveryService((host, port)) as rsd:
        with DvtSecureSocketProxyService(rsd) as dvt:
            self.monitor_logic(dvt)
except Exception as exc:
    logger.exception(f'An error occurred executing {self.__class__.__name__}. {exc}')

Hi @doronz88, When I use the above logic I get this error almost all times:

[2023-09-23 10:11:51,204] [ERROR] generic_monitor.py -> monitor_thread line:41 An error occurred executing IOSPidMonitor. Got RstStreamFrame(stream_id=1, flags=[]): error_code=5
Traceback (most recent call last):
  File "/Users/juan/Documents/Developer/apptim-agent/apptim_agent/profiler/ios/monitors/generic_monitor.py", line 29, in monitor_thread
    with RemoteServiceDiscoveryService((host, port)) as rsd:
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remote_service_discovery.py", line 90, in __enter__
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remote_service_discovery.py", line 37, in connect
    self.peer_info = self.service.receive_response()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remotexpc.py", line 88, in receive_response
    frame = self._receive_next_data_frame()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remotexpc.py", line 155, in _receive_next_data_frame
    raise StreamClosedError(f'Got {frame}')
pymobiledevice3.exceptions.StreamClosedError: Got RstStreamFrame(stream_id=1, flags=[]): error_code=5

What could be the problem here?

Edit: I have added a retry logic

try:
    if self.device.version_is_17_or_greater():
        for attempt in range(1, max_retries + 1):
            host, port = ("fd56:d87a:8e52::1", 54336)
            with RemoteServiceDiscoveryService((host, port)) as rsd:
                with DvtSecureSocketProxyService(rsd) as dvt:
                    self.monitor_logic(dvt)
except StreamClosedError as exc:
    logger.warning(f'Connection attempt {attempt} failed: {exc}')
    if attempt < max_retries:
        logger.info(f'Retrying in {retry_delay} seconds...')
        time.sleep(retry_delay)
    else:
        logger.error('Max retries reached. Unable to establish a connection.')
        break
except Exception as exc:
    logger.exception(f'An error occurred executing {self.__class__.__name__}. {exc}')

and based on logs, apparently always fails at least 1 time:

[2023-09-23 10:28:07,431] [WARNING] generic_monitor.py -> monitor_thread line:46 Connection attempt 1 failed: Got RstStreamFrame(stream_id=1, flags=[]): error_code=5
[2023-09-23 10:28:07,431] [INFO] generic_monitor.py -> monitor_thread line:48 Retrying in 5 seconds...

Originally posted by @morellexf13 in https://github.com/doronz88/pymobiledevice3/issues/562#issuecomment-1732316291

[doronz88]

It seens the device decided to close your connection. I'm familiar with it happening only during browse for new devices - that's exactly what the stop_remoted() context-manager is for.

Please try to close any instance of XCode/Instruments.

[morellexf13]

Oh, makes sense since I'm literally polling for devices while profiling!

[doronz88]

Any updates? Closing the Xcode related stuff solved this?

[morellexf13]

The problem persists, actually the only way for me to make this work is by retrying

[doronz88]

I don't have an idea how to reproduce your problem. Does this error also occur from pymobiledevice3 builtin CLI aswell?

[morellexf13]

I didn't try to reproduce this on the CLI, has to be something about polling all these different services at the same time.

[doronz88]

I believe I discovered the cause (though wasn't able to reproduce). Just created #572 which should solve this. In general I added the two: stop_remoted_if_required() & resume_remoted_if_required() to stop remoted from doing anything while the tunnel is still being established.

Please update if this fix solves the issue

[morellexf13]

I still have this problem, could be something about handling multiple RemoteServiceDiscoveryService instances like this?

def connection_context(self, callback):
    if self.version_is_17_or_greater():
        # Use Remote XPC Tunnel Address and Port
        remote_tunnel = RemoteXPC().get_rsd()
        if remote_tunnel:
            host, port = (remote_tunnel["address"], remote_tunnel["port"])
            with RemoteServiceDiscoveryService((host, port)) as rsd:
                with DvtSecureSocketProxyService(rsd) as dvt:
                    callback(dvt)
        else:
            logger.error("An error occurred getting XPC rsd...")
    else:
        # Use USB Mux
        lockdown = self.get_lockdown()
        with DvtSecureSocketProxyService(lockdown=lockdown) as dvt:
            callback(dvt)

def monitor_thread(self):
    while base_profiler.running:
        try:
            self.device.connection_context(self.monitor_logic)
        except StreamClosedError:
            pass
        except Exception as exc:
            logger.exception(f'An error occurred executing {self.__class__.__name__}. {exc}')

  def monitor_logic(self, dvt):
      # When creating a new monitor you may want to override this method with your
      # own logic
      pass

    -----------------------------------
    **Examples**
    class IOSDisplayMonitor(IOSGenericMonitor):

    def monitor_logic(self, dvt):
        self.file_manager.create_tsv_from_model(IOSFPSModel())
        with Graphics(dvt) as graphics:
            for stats in graphics:

    **The same for each of these**      
    self.monitor_system()
    self.monitor_display()
    self.monitor_syslog()
    self.monitor_xcode_energy()

[doronz88]

This code doesn't show how you handled the remoted stop while creating the tunnel

[morellexf13]

Sorry, here it is: https://github.com/morellexf13/ios-remote-xpc-tunnel-creator/ Basically, we have a binary that handles all of that and then in our code we call this applescript with given permissions.

[doronz88]

Ok so I misunderstood. The exception occurs NOT during the tunnel creation, but using the newly created one. I'm failing to reproduce it and cannot find a reason why this would happen. If you can find the reason please update.

I this should happen as you described ONLY when trying to connect to the untrsuted RSD.

[morellexf13]

Traceback (most recent call last):
  File "/Users/juan/Documents/Developer/apptim-agent/apptim_agent/profiler/ios/monitors/generic_monitor.py", line 28, in monitor_thread
    with RemoteServiceDiscoveryService((host, port)) as rsd:
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remote_service_discovery.py", line 95, in __enter__
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remote_service_discovery.py", line 41, in connect
    self.peer_info = self.service.receive_response()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remotexpc.py", line 88, in receive_response
    frame = self._receive_next_data_frame()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remotexpc.py", line 151, in _receive_next_data_frame
    frame = self._receive_frame()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remotexpc.py", line 169, in _receive_frame
    buf = self._recvall(FRAME_HEADER_SIZE)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymobiledevice3/remote/remotexpc.py", line 178, in _recvall
    chunk = self.sock.recv(size - len(data))
ConnectionResetError: [Errno 54] Connection reset by peer

All I can see differently is this other error. I can't find the cause yet :/

[morellexf13]

Update: I was using just ONE rsd address to handle all the monitors. Programatically:

with RemoteServiceDiscoveryService(rsd_address) as rsd:
  with DvtSecureSocketProxyService(rsd) as dvt:
      callback(dvt)

rsd here was ALWAYS the same only address for each of these contexts (we have 5 monitors running at the same time). creating one rsd to use per each monitor works fine without problems!

[doronz88]

Okay I finally got what the problem was (and was able to reproduce thanks to that). It's not a bug in in pmd3, but the same issue as with the remoted. Each tunnel is received at a different remoted endpoint on the device side. When performing the RemoteServiceDiscoveryService handshake it will cause random connection closes since each endpoint can only handle a single RSD instance. This is why we are stopping the Mac's own remoted. On your case, each two tunnel usages were competing over it.

This can be considered a bug in remoted or just a poor design choice. Either case, I'm glad we figured this out.