search

unbit / uwsgi

uWSGI application server container
http://projects.unbit.it/uwsgi
Other
3.45k stars 686 forks source link

Using gevent plugin suppresses thunder-lock #1757

Open RussellLuo opened 6 years ago

RussellLuo commented 6 years ago

Phenomenon

1. Using thunder-lock alone works well

$ uwsgi --http-socket 0.0.0.0:9090 --wsgi-file foobar.py --master --processes 2 --thunder-lock
*** Starting uWSGI 2.0.17 (32bit) on [Sun Mar 11 08:13:33 2018] ***
compiled with version: 4.8.2 on 08 March 2018 07:35:49
os: Linux-3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:44 UTC 2014
...
lock engine: pthread robust mutexes
thunder lock: enabled
...
spawned uWSGI master process (pid: 8267)
spawned uWSGI worker 1 (pid: 8268, cores: 1)
spawned uWSGI worker 2 (pid: 8269, cores: 1)
$ strace -p 8268
Process 8268 attached
futex(0xb6f16000, FUTEX_UNLOCK_PI, -1225691136) = 0
futex(0xb6f16000, FUTEX_LOCK_PI, 1)     = 0
epoll_wait(4, {{EPOLLIN, {u32=3, u64=3}}}, 1, -1) = 1
accept4(3, {sa_family=AF_INET, sin_port=htons(52016), sin_addr=inet_addr("125.69.20.18")}, [16], SOCK_NONBLOCK) = 6
...
$ strace -p 8269
Process 8269 attached
futex(0xb6f16000, FUTEX_UNLOCK_PI, -1225691136) = 0
futex(0xb6f16000, FUTEX_LOCK_PI, 1)     = 0
epoll_wait(4,

2. Combining gevent with thunder-lock does not work

$ uwsgi --http-socket 0.0.0.0:9090 --wsgi-file foobar.py --master --processes 2 --thunder-lock --gevent 2
*** Starting uWSGI 2.0.17 (32bit) on [Sun Mar 11 08:40:03 2018] ***
compiled with version: 4.8.2 on 08 March 2018 07:35:49
os: Linux-3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:44 UTC 2014
...
lock engine: pthread robust mutexes
thunder lock: enabled
...
spawned uWSGI master process (pid: 8376)
spawned uWSGI worker 1 (pid: 8377, cores: 2)
spawned uWSGI worker 2 (pid: 8378, cores: 2)
*** running gevent loop engine [addr:0x80deac0] ***
$ strace -p 8377
Process 8377 attached
clock_gettime(CLOCK_MONOTONIC, {91266079, 199837057}) = 0
gettimeofday({1520772012, 327530}, NULL) = 0
clock_gettime(CLOCK_MONOTONIC, {91266079, 199917756}) 
epoll_wait(4, {{EPOLLIN, {u32=3, u64=4294967299}}}, 64, 59743) = 1
clock_gettime(CLOCK_MONOTONIC, {91266797, 809083555}) = 0
gettimeofday({1520772730, 936800}, NULL) = 0
accept4(3, {sa_family=AF_INET, sin_port=htons(50216), sin_addr=inet_addr("125.69.20.18")}, [16], SOCK_NONBLOCK) = 7
...
$ strace -p 8378
Process 8378 attached
clock_gettime(CLOCK_MONOTONIC, {91266093, 89470893}) = 0
gettimeofday({1520772026, 217616}, NULL) = 0
clock_gettime(CLOCK_MONOTONIC, {91266093, 90181244}) = 0
epoll_wait(4, {{EPOLLIN, {u32=3, u64=4294967299}}}, 64, 59743) = 1
clock_gettime(CLOCK_MONOTONIC, {91266797, 809322672}) = 0
gettimeofday({1520772730, 937077}, NULL) = 0
accept4(3, 0xb69da840, [110], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
...

Question

marc1n commented 6 years ago

Related issue: https://github.com/unbit/uwsgi/issues/1605

In gevent (and other async modes) main loop engine actually does not use thunder-lock (there is no thunder_lock macro call). I don't know why.

Only normal ("simple") and "rbthreads" loop engines are using thunder_lock.

  1. Simple loop code (simple_loop_run function) is here: https://github.com/unbit/uwsgi/blob/2.0.17/core/loop.c#L117

    Function simple_loop_run calls wsgi_req_accept (https://github.com/unbit/uwsgi/blob/2.0.17/core/utils.c#L1484) which calls thunder_lock macro.

  2. Async loop code (async_loop) is here: https://github.com/unbit/uwsgi/blob/2.0.17/core/async.c#L408

    Function async_loop (and asyncio_loop, tornado_loop etc) calls wsgi_req_simple_accept (https://github.com/unbit/uwsgi/blob/2.0.17/core/utils.c#L1453) which does not call thunder_lock macro.