[udf] Unlock Lua for user-defined functions

[amotl]

Hi again,

similar to what Webscript has been doing for HTTP, and @progrium explored on behalf of Airscript the other day, this patch explores the feasibility of "using Lua for MQTT scripting".

Thanks to the excellent lupa package by @scoder 💯, the patch is ridiculously small again, similar to the other one GH-667, which brings in JavaScript for the same reasons. Both patches are making mqttwarn a host for other languages, effectively bringing together the MQTT protocol, mqttwarn's transformation engine, and now, more languages than just Python, into a small-scale framework it is so much fun to fiddle with.

I think in the current era of serverless computing and FaaS systems, often operated by large cloud service vendors, it is so important to have open source alternatives around, which, more often than not, can do things more efficiently and conveniently in terms of development efforts, and also don't have several layers of indirection between caller and callee, omitting all the dynamic scaling and throttling aspects usually present in large application container / language host infrastructures, which I found to have a significant cost, both mentally and operationally.

Maybe it works well when it's deployed well, but well, long story short, I prefer personal computing over needing to wrap my head around how to invoke services/programs/functions on 3rd-party infrastructure, which is mostly not completely transparent, or impossible to run on a personal computer at all.

My motivation is mainly but not exclusively driven by my bad experiences in the past when I have been sucked into analyzing subtle errors on shared cloud infrastructure. It's ridiculous, and I am so happy to have learned how to operate my own machines for hosting and publishing digital goods. Enjoy reading! [^1]

• https://github.com/confluentinc/librdkafka/issues/3109

The documentation of the new feature reflects my experiences, and its preview is available here:

• https://mqttwarn--669.org.readthedocs.build/en/669/configure/transformation.html#using-other-languages
• https://mqttwarn--669.org.readthedocs.build/en/669/examples/owntracks-ntfy/readme-variants.html#lua

With kind regards, Andreas.

[^1]: And no offense to the Azure and Kafka authors, engineers and users, I know I am comparing apples with oranges here, you are doing an excellent job, and nobody would want to professionally run Kafka on personal computers anyway ;]. It's all just my personal idea of a small-scale FaaS framework with an adapter to a message bus system - here, MQTT.

[scoder]

Might be worth mentioning that Lupa 2.0 is in the making (no release date yet), which provides different Lua implementations in separate modules, i.e. lupa.lua53, lupa.lua54, lupa.luajit21, etc., all having the same API as before. You could then be specific about the Lua version that you want to provide, just with an import. Or, you could let users decide, e.g. by looking at the shebang line of a script. That's probably out of scope for this PR, but worth keeping an eye on for later.

[amotl]

Hi Stefan,

thanks for the heads up. I had the impression that I was using Lupa 2.0 already. It works very well, thank you so much!

$ pip install lupa
Collecting lupa
  Using cached lupa-2.0.tar.gz (6.3 MB)

$ python -c 'import lupa; print(lupa.__version__)'
2.0

I recognized that possibility to invoke different interpreters already -- very impressive! -- but you are right, I did not look into making this configurable yet, and was happy that Lupa just works out of the box. But, very true, unlocking LuaJIT with a subsequent iteration very much resonates with me for this use case.

With kind regards, Andreas.

[scoder]

I had the impression that I was using Lupa 2.0 already.

Ah, right, sorry. When you maintain multiple packages, it's easy to forget which is released at which version (or not yet).

The simple "import lupa" obviously still works and gives you the latest available Lua version. That might not be the best choice for an embedded scripting engine, since a new lupa release might also change that version. You might want to keep the version import explicit in your usage code, and change it at your own pace.

It works very well, thank you so much!

Happy to read that.

[codecov[bot]]

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (8cd99f7) 49.96% compared to head (7d2fc46) 50.12%.

Files	Patch %	Lines
mqttwarn/util.py	80.95%	4 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## amo/javascript #669 +/- ## ================================================== + Coverage 49.96% 50.12% +0.15% ================================================== Files 81 81 Lines 4051 4072 +21 ================================================== + Hits 2024 2041 +17 - Misses 2027 2031 +4 ``` | [Flag](https://app.codecov.io/gh/mqtt-tools/mqttwarn/pull/669/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mqtt-tools) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/mqtt-tools/mqttwarn/pull/669/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mqtt-tools) | `50.12% <80.95%> (+0.15%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mqtt-tools#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.