Hybrid programming in Envoy (Make Lua-resty-FFI run on Envoy)

[kingluo]

lua-resty-ffi provides an efficient and generic API to do hybrid programming in openresty/envoy with mainstream languages.

https://github.com/kingluo/lua-resty-ffi

lua-resty-ffi is now available on envoy, which means you can use Go, Python, Java, Rust and NodeJS to develop new features for envoy, and existing lua-resty-ffi based libraries such as lua- resty-ffi-grpc can be switched over to envoy without any code changes.

http://luajit.io/posts/make-lua-resty-ffi-run-on-envoy/

Welcome any suggestions or discussions! Thanks.

[alyssawilk]

cc @mattklein123 @wbpcode @phlax

[wbpcode]

cc @StarryVae

[wbpcode]

This sounds great to me. But our initial target of lua extension is to create simple script. Not sure if is acceptable to create a new ffi-based lua extension. Will we refactor current CAPI-based lua extension and how to ensure the back-compatibility?

[kingluo]

This sounds great to me. But our initial target of lua extension is to create simple script. Not sure if is acceptable to create a new ffi-based lua extension. Will we refactor current CAPI-based lua extension and how to ensure the back-compatibility?

Maybe both coexist? CAPI is suitable to call on specific objects, although it's less efficient than ffi, but ffi requires extern C symbol.

[kingluo]

Side question: currently the AsyncClient needs upstream cluster definition, not so dynamic configurable like openresty co-socket, which can specify destination address. Is it feasible and necessary to implement lua co-socket based on IoSocketHandleImpl?

Similarly, could we implement lua timer via dispatcher?

Yes, I know the lua filter is not supposed to be so complicated, but the current implementation lacks of important functionalities like co-socket and timer, which limits its usage.

[wbpcode]

sgtm.

[wbpcode]

But a more detailed proposal is necessary and we can review or discuss it in more detail before we start the work. 😺

[kingluo]

But a more detailed proposal is necessary and we can review or discuss it in more detail before we start the work. 😺

Sure, I'll write a proposal later, which will involve several topics about the lua filter extensions.

Please review my code changes if you have time, especially the code style convention of envoy. Thanks.

https://github.com/envoyproxy/envoy/compare/main...kingluo:envoy:lua-resty-ffi

[kingluo]

Hi, I have a new blog article about asynchronous filter processing through goroutine, comparing the performance difference of using lua filter (with lua-resty-ffi) and golang filter respectively. If you are interested, take a look please, thank you! :-)

http://luajit.io/posts/envoy-async-http-filter-lua-resty-ffi-vs-golang/

cc @mattklein123 @wbpcode @phlax

[phlax]

great post! thanks