eandersson / amqpstorm

Thread-safe Python RabbitMQ Client & Management library
https://www.amqpstorm.io/
MIT License
186 stars 36 forks source link

100% cpu usage problem #110

Closed aiden0z closed 2 years ago

aiden0z commented 2 years ago

The _process_incoming_data may cause 100% cpu usage problem when read data from bad socket fd.

    def _process_incoming_data(self):
        """Retrieve and process any incoming data.
        :return:
        """
        while self._running.is_set():
            if self.poller.is_ready:
                self.data_in += self._receive()
                self.data_in = self._on_read_impl(self.data_in)
  1. py-spy output

image

  1. get the problem thread

image

  1. strace found select syscall return immediately and recvfrom syscall read zero data.

image

We should check the received data length to fix the dead loop problem.

Ref: The Redis client how to check the received data length. https://github.com/redis/redis-py/blob/16cfcc7fced84d2b53edf95af1c40b230b30fc3d/redis/connection.py#L435

eandersson commented 2 years ago

Thanks @aiden0z !