splunk / splunk-sdk-python

Splunk Software Development Kit for Python
http://dev.splunk.com
Apache License 2.0
698 stars 370 forks source link

High CPU iterating over ResponseReader (python3) #287

Closed abotsis closed 5 years ago

abotsis commented 5 years ago

Howdy, I'm seeing high cpu iterating over a ResponseReader object in Python 3.7.

When doing something like:

results = service.jobs.export('search *')
for _ in results:
    pass

CPU spikes to 100%, and it's slow. cProfile indicates (trimmed):

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.013    0.013  174.869  174.869 <string>:1(<module>)
 26612304   11.754    0.000   89.386    0.000 binding.py:1285(peek)
 53224608   35.328    0.000  146.927    0.000 binding.py:1304(read)
 53224608   59.029    0.000  107.549    0.000 client.py:436(read)
 53224607   27.661    0.000   43.139    0.000 client.py:468(readinto)

A little digging reveals two potential performance issues with the ResultsReader.read() method:

  1. It's reading one character at a time.
  2. For each character read, it appends to self._buffer

Additionally, adding a few print statements to read() immediately before return shows this pattern:

DEBUG read: 1 size: 0
DEBUG read: 1 size: 1
DEBUG read: 1 size: 0
DEBUG read: 1 size: 1
DEBUG read: 1 size: 0
DEBUG read: 1 size: 1
DEBUG read: 1 size: 0
DEBUG read: 1 size: 1
DEBUG read: 1 size: 0
DEBUG read: 1 size: 1
DEBUG read: 1 size: 0
DEBUG read: 1 size: 1
DEBUG read: 1 size: 0
DEBUG read: 1 size: 1
DEBUG read: 1 size: 0

In the example above, if I instead iterate over results._response, things work as expected. I have not tried to reproduce in Python 2.

As a side note, while I'm not familiar with the implementation of this in it's entirety, I've noticed:

    def read(self, size = None):
        r = self._buffer
        self._buffer = b''
        if size is not None:
            size -= len(r)
        r = r + self._response.read(size)
        return r

if size=0 (which it does sometimes, apparently) and self._buffer==0, this will pass -1 to the self._response.read call.

shakeelmohamed commented 5 years ago

Seems like a dupe of #223 - there's some recommended workarounds there. Please re-open if this is different