search

couchbase / couchbase-python-client

Couchbase Python Client Library (Official)
https://www.couchbase.com/
Apache License 2.0
243 stars 110 forks source link

slow http #3

Closed qyloxe closed 2 months ago

qyloxe commented 13 years ago

in res_client.py in method _http_request you are using httplib2.Http().request(api, method, params, headers)

it is dozens to hundreds times slower than using urllib (urllib2, urllib3) because there's inefficiency in method readline (in socket.py). this function is linear to the length of the message because of working byte by byte:

 392 :     2300 :       0.031000137329 : 0.0000134783 |                 data = None
 393 :     2300 :       0.015000104904 : 0.0000065218 |                 recv = self._sock.recv
 394 :    43183 :       0.311999559402 : 0.0000072251 |                 while data != "\n":
 395 :    40883 :       0.124999761581 : 0.0000030575 |                     data = recv(1)
 396 :    40883 :      34.978999853134 : 0.0008555879 |                     if not data:
 397 :          :                      :              |                         break
 398 :    40883 :       0.263999938965 : 0.0000064575 |                     buffers.append(data)
 399 :     2300 :       0.016000032425 : 0.0000069565 |                 return "".join(buffers)

look above - 34.97 - it is a time in seconds for getting 180 keys from couchbase. 40883 - it is an execution count. with urllib.urlopen (or similar) this time reduces to about 3 seconds.