Kong / ngx_wasm_module

Nginx + WebAssembly
Apache License 2.0
79 stars 7 forks source link

refactor(lua-bridge) rewrite for full support of yielding Lua threads #539

Closed thibaultcha closed 4 months ago

thibaultcha commented 4 months ago

Major refactor of the Lua bridge to support multiple concurrent yielding Lua threads. The old implementation would break down when scheduling more than one yielding Lua thread at a time.

The new implementation "tricks" OpenResty by scheduling uthreads via C and passing these threads to the OpenResty runloop as if they were created from Lua (via ngx.thread). Because all uthreads must resume their "parent thread" when finished (as per OpenResty's implementation), we schedule a stub "entry thread" whenever we are trying to use the Lua bridge. This entry thread itself does nothing and is collected at request pool cleanup.

List of significant changes for this refactor:

Fix https://github.com/Kong/ngx_wasm_module/issues/524

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 89.11392% with 43 lines in your changes are missing coverage. Please review.

Project coverage is 90.45255%. Comparing base (b19d405) to head (8fd1fcb).

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539/graphs/tree.svg?width=650&height=150&src=pr&token=T0PT2Q9IAN&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong)](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong) ```diff @@ Coverage Diff @@ ## main #539 +/- ## =================================================== + Coverage 90.05848% 90.45255% +0.39406% =================================================== Files 47 47 Lines 10089 10275 +186 =================================================== + Hits 9086 9294 +208 + Misses 1003 981 -22 ``` | [Files](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong) | Coverage Δ | | |---|---|---| | [src/common/lua/ngx\_wasm\_lua\_ffi.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fcommon%2Flua%2Fngx_wasm_lua_ffi.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL2NvbW1vbi9sdWEvbmd4X3dhc21fbHVhX2ZmaS5j) | `92.92035% <100.00000%> (-0.67965%)` | :arrow_down: | | [src/common/lua/ngx\_wasm\_lua\_resolver.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fcommon%2Flua%2Fngx_wasm_lua_resolver.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL2NvbW1vbi9sdWEvbmd4X3dhc21fbHVhX3Jlc29sdmVyLmM=) | `76.66667% <100.00000%> (+12.83687%)` | :arrow_up: | | [src/common/ngx\_wasm\_socket\_tcp\_readers.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fcommon%2Fngx_wasm_socket_tcp_readers.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL2NvbW1vbi9uZ3hfd2FzbV9zb2NrZXRfdGNwX3JlYWRlcnMuYw==) | `87.81513% <100.00000%> (ø)` | | | [src/common/proxy\_wasm/ngx\_proxy\_wasm.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fcommon%2Fproxy_wasm%2Fngx_proxy_wasm.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL2NvbW1vbi9wcm94eV93YXNtL25neF9wcm94eV93YXNtLmM=) | `92.58850% <100.00000%> (+0.04952%)` | :arrow_up: | | [src/http/ngx\_http\_wasm\_filter\_module.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fhttp%2Fngx_http_wasm_filter_module.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL2h0dHAvbmd4X2h0dHBfd2FzbV9maWx0ZXJfbW9kdWxlLmM=) | `92.85714% <100.00000%> (+0.11970%)` | :arrow_up: | | [src/http/proxy\_wasm/ngx\_http\_proxy\_wasm\_dispatch.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fhttp%2Fproxy_wasm%2Fngx_http_proxy_wasm_dispatch.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL2h0dHAvcHJveHlfd2FzbS9uZ3hfaHR0cF9wcm94eV93YXNtX2Rpc3BhdGNoLmM=) | `91.29353% <100.00000%> (+0.06546%)` | :arrow_up: | | [src/wasm/ngx\_wasm.h](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fwasm%2Fngx_wasm.h&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL3dhc20vbmd4X3dhc20uaA==) | `100.00000% <ø> (ø)` | | | [src/wasm/ngx\_wasm\_ops.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fwasm%2Fngx_wasm_ops.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL3dhc20vbmd4X3dhc21fb3BzLmM=) | `92.73504% <100.00000%> (+0.60540%)` | :arrow_up: | | [src/common/ngx\_wasm\_socket\_tcp.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fcommon%2Fngx_wasm_socket_tcp.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL2NvbW1vbi9uZ3hfd2FzbV9zb2NrZXRfdGNwLmM=) | `89.34993% <80.95238%> (+0.52233%)` | :arrow_up: | | [src/wasm/ngx\_wasm\_core\_host.c](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree&filepath=src%2Fwasm%2Fngx_wasm_core_host.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#diff-c3JjL3dhc20vbmd4X3dhc21fY29yZV9ob3N0LmM=) | `86.53846% <78.26087%> (+12.72893%)` | :arrow_up: | | ... and [2 more](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong) | | | [Flag](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong) | Coverage Δ | | |---|---|---| | [unit](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong) | `90.19303% <88.01170%> (+0.04185%)` | :arrow_up: | | [valgrind](https://app.codecov.io/gh/Kong/ngx_wasm_module/pull/539/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong) | `81.25601% <90.11628%> (+3.24058%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kong#carryforward-flags-in-the-pull-request-comment) to find out more.
thibaultcha commented 4 months ago

Moved to #546.