Gevent raises timeout

[lukebakken]

Environment:

$ python --version
Python 2.7.11

$ uname -a
Linux brahms 4.4.5-1-ARCH #1 SMP PREEMPT Thu Mar 10 07:38:19 CET 2016 x86_64 GNU/Linux

$ pip list
cornice (1.2.0)
gevent (1.1.0)
greenlet (0.4.9)
http-parser (0.8.3)
PasteDeploy (1.5.2)
pip (8.1.0)
pyramid (1.6.1)
repoze.lru (0.6)
requests (2.9.1)
setuptools (20.2.2)
simplejson (3.8.2)
statsd-client (1.0.4)
translationstring (1.3)
vaurien (1.9)
vaurienclient (1.1)
venusian (1.0)
virtualenv (15.0.0)
WebOb (1.5.1)
wheel (0.29.0)
zope.deprecation (4.1.2)
zope.interface (4.1.3)

Command:

$ vaurien --protocol http --proxy 0.0.0.0:8080 --backend www.bakken.io:80 --behavior 100:dummy
2016-03-13 18:07:00 [40688] [INFO] Starting the Chaos TCP Server
2016-03-13 18:07:00 [40688] [INFO] Options:
2016-03-13 18:07:00 [40688] [INFO] * proxies from 0.0.0.0:8080 to www.bakken.io:80
2016-03-13 18:07:00 [40688] [INFO] * timeout: 30
2016-03-13 18:07:00 [40688] [INFO] * stay_connected: 0
2016-03-13 18:07:00 [40688] [INFO] * pool_max_size: 100
2016-03-13 18:07:00 [40688] [INFO] * pool_timeout: 30
2016-03-13 18:07:00 [40688] [INFO] * async_mode: 1

Exception raised:

Traceback (most recent call last):
  File "/home/lbakken/.pyenv/versions/2.7.11/envs/vaurien-2.7.11/lib/python2.7/site-packages/gevent/greenlet.py", line 534, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/lbakken/.pyenv/versions/2.7.11/envs/vaurien-2.7.11/lib/python2.7/site-packages/vaurien/proxy.py", line 220, in _weirdify
    behavior, behavior_name)
  File "/home/lbakken/.pyenv/versions/2.7.11/envs/vaurien-2.7.11/lib/python2.7/site-packages/vaurien/proxy.py", line 168, in _weirdify
    return self.handler(source, dest, to_backend, behavior)
  File "/home/lbakken/.pyenv/versions/2.7.11/envs/vaurien-2.7.11/lib/python2.7/site-packages/vaurien/protocols/base.py", line 58, in __call__
    return self._handle(source, dest, to_backend)
  File "/home/lbakken/.pyenv/versions/2.7.11/envs/vaurien-2.7.11/lib/python2.7/site-packages/vaurien/protocols/http.py", line 46, in _handle
    dest.sendall(data)
  File "/home/lbakken/.pyenv/versions/2.7.11/envs/vaurien-2.7.11/lib/python2.7/site-packages/gevent/_socket2.py", line 412, in sendall
    timeleft = self.__send_chunk(chunk, flags, timeleft, end)
  File "/home/lbakken/.pyenv/versions/2.7.11/envs/vaurien-2.7.11/lib/python2.7/site-packages/gevent/_socket2.py", line 357, in __send_chunk
    raise timeout('timed out')
timeout: timed out
<Greenlet at 0x7fa4d8a547d0: <bound method RandomProxy._weirdify of <RandomProxy at 0x7fa4d8f43590 fileno=5 address=0.0.0.0:8080>>(<socket at 0x7fa4d8600210 fileno=7 sock=192.168.13, <socket at 0x7fa4d8600190 fileno=8 sock=192.168.13, Tr
ue, 'http.755e4232-c732-49d0-ab0c-d7a501c21854.', <vaurien.behaviors.dummy.Dummy object at 0x7fa4d8f, 'dummy')> failed with timeout

PR on the way.

[cchantep]

Anything new? With this issue, vaurien is just not working.

[andrewmichaelsmith]

Hi, I'm new to vaurien are there any work arounds for this?

I've tried a variety of different configs (including sync, reconnect) but it always seems to fail. Using vaurien 1.9. Also tried with 1.9 but hitting the same issue.

[andrewmichaelsmith]

In case this helps:

root@2f2edc0464b0:/# python
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import vaurien
>>> vaurien.__version__
'1.8'
>>> 

root@2f2edc0464b0:/# vaurien --protocol http --proxy 0.0.0.0:5672 --backend google.com:80         --behavior 20:delay                                                                                                                                                                                                  
2016-05-13 14:10:28 [18] [INFO] Starting the Chaos TCP Server
2016-05-13 14:10:28 [18] [INFO] Options:
2016-05-13 14:10:28 [18] [INFO] * proxies from 0.0.0.0:5672 to google.com:80
2016-05-13 14:10:28 [18] [INFO] * timeout: 30
2016-05-13 14:10:28 [18] [INFO] * stay_connected: 0
2016-05-13 14:10:28 [18] [INFO] * pool_max_size: 100
2016-05-13 14:10:28 [18] [INFO] * pool_timeout: 30
2016-05-13 14:10:28 [18] [INFO] * async_mode: 1
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 534, in run
    result = self._run(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/proxy.py", line 214, in _weirdify
    behavior, behavior_name)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/proxy.py", line 162, in _weirdify
    return self.handler(source, dest, to_backend, behavior)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/protocols/base.py", line 58, in __call__
    return self._handle(source, dest, to_backend)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/protocols/http.py", line 46, in _handle
    dest.sendall(data)
  File "/usr/local/lib/python2.7/dist-packages/gevent/_socket2.py", line 412, in sendall
    timeleft = self.__send_chunk(chunk, flags, timeleft, end)
  File "/usr/local/lib/python2.7/dist-packages/gevent/_socket2.py", line 357, in __send_chunk
    raise timeout('timed out')
timeout: timed out
<Greenlet at 0x7f559fc8bcd0: <bound method RandomProxy._weirdify of <RandomProxy at 0x7f559fbf5590 fileno=5 address=0.0.0.0:5672>>(<socket at 0x7f559f971a50 fileno=6 sock=172.17.138, <socket at 0x7f559f971cd0 fileno=8 sock=172.17.138, True, 'http.97be5caf-30b7-43d9-8143-7d457b34d8d9.', <vaurien.behaviors.dummy.Dummy object at 0x7f559fb, 'dummy')> failed with timeout

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 534, in run
    result = self._run(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/gevent/baseserver.py", line 25, in _handle_and_close_when_done
    return handle(*args_tuple)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/proxy.py", line 116, in handle
    res = [green.get() for green in greens]
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 480, in get
    self._raise_exception()
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 171, in _raise_exception
    reraise(*self.exc_info)
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 534, in run
    result = self._run(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/proxy.py", line 214, in _weirdify
    behavior, behavior_name)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/proxy.py", line 162, in _weirdify
    return self.handler(source, dest, to_backend, behavior)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/protocols/base.py", line 58, in __call__
    return self._handle(source, dest, to_backend)
  File "/usr/local/lib/python2.7/dist-packages/vaurien/protocols/http.py", line 46, in _handle
    dest.sendall(data)
  File "/usr/local/lib/python2.7/dist-packages/gevent/_socket2.py", line 412, in sendall
    timeleft = self.__send_chunk(chunk, flags, timeleft, end)
  File "/usr/local/lib/python2.7/dist-packages/gevent/_socket2.py", line 357, in __send_chunk
    raise timeout('timed out')
timeout: timed out
<Greenlet at 0x7f559fc8bc30: _handle_and_close_when_done(<bound method RandomProxy.handle of <RandomProxy a, <bound method RandomProxy.do_close of <RandomProxy, (<socket at 0x7f559f971a50 fileno=[Errno 9] Bad fi)> failed with timeout

[lukebakken]

@andrewmichaelsmith - see PR #53

[andrewmichaelsmith]

@lukebakken thanks! I just installed your branch and it works perfectly.