Due to LuaJIT, Plugins aren't compatible with Lua 5.1 even if they don't require LuaJIT

[juru1234]

Problem

Hi, we are in a situation where lua plugins use e.g. goto's in their code that isn't compatible with Lua 5.1 because 99% use neovim with LuaJIT that does support goto's. Because of this, these plugins cannot be used with neovim without LuaJIT even if they don't require LuaJIT at all. Do you still consider neovim with Lua 5.1 as the default under these circumstances? I think this affects ppc64el, riscv64, and s390x because they currently do not support LuaJIT, and PR's are ignored by the LuaJIT maintainer (https://github.com/LuaJIT/LuaJIT/pull/631).

E.g. the fzf-lua maintainer thankfully removed all the goto's in his code lately because he was just not aware that his code wasn't compatible without LuaJIT even if it doesn't require LuaJIT: https://github.com/ibhagwan/fzf-lua/commit/e5c14900e8abaa962f6880853ce0f749b511aa85

Steps to reproduce

Use neovim without LuaJIT with e.g. a plugin that uses goto's

Expected behavior

If neovim without LuaJIT is still seen as the default, neovim should support Lua 5.2 so that code works, that people wrote using neovim with LuaJIT (code that doesn't require LuaJIT at all). Switching to luajit2 instead of luajit could also be an option.

Neovim version (nvim -v)

0.10.0

Vim (not Nvim) behaves the same?

no

Operating system/version

Fedora

Terminal name/version

alacritty

$TERM environment variable

xterm-256color

Installation

system package manager

[clason]

Do you still consider neovim with Lua 5.1 as the default under these circumstances?

Yes.

If neovim without LuaJIT is still seen as the default, neovim should support Lua 5.2 so that code works

No. Lua 5.2 is a different language and would cut out LuaJIT (which even though it cannot be the baseline due to lack of support for all our platforms, it's vastly preferred for performance reasons).

Switching to luajit2 instead of luajit could also be an option.

It's not; this has been discussed before (search closed issues and PRs).

[bfredl]

Also, distros for these platforms can just build neovim linked with luajit2, and some already do. no "switching" needs to be done (any fork can be used by downstream packagers as long as it retains lua5.1 compat including C ABI)

[juru1234]

As far as I know, there is no distro that ships neovim with luajit2 for specific architectures. They said, that they don't link packages to other packages dependent on the architecture. What some distros do is shipping neovim with the luajit patch from here: https://github.com/LuaJIT/LuaJIT/pull/631

But sadly, this patch isn't accepted upstream.

If you say that distros should link to luajit2, then you cannot consider neovim without luajit as the default anymore.

[clason]

If you say that distros should link to luajit2, then you cannot consider neovim without luajit as the default anymore.

I don't see how that follows. And nobody said distros should do that, just that they could (because we only require Lua 5.1 compat -- this is intentional). The point is that they can build Neovim against any 5.1 compatible Lua interpreter, exactly so they can pick the best one they can get (for performance or platform support, and they can use different interpreters for different platforms).

If distros don't want to do that, that's between them and their users; nothing we can do about it.

[bfredl]

All i am saying is that distros do not need our permission to ship Neovim with a working luajit on these systems. Distros adding patches to fix packages is a rule, not somehow an exception.

If you say that distros should link to luajit2, then you cannot consider neovim without luajit as the default anymore.

It is the documented minimal supported version. . I don't see any reason to pretend there is anything not clear about this.

[dundargoc]

Related discussion: https://github.com/neovim/neovim/pull/24313

I think the reason the previous issue was closed was partly because of the reasoning that this is something that should be fixed by luajit instead of pathed by us. The linked PR makes me think this is no longer valid as @MikePall does not seem interested in supporting s390x for whatever reason. Unsure if the premise change is important enough to warrant a re-discussion on luajit vs luajit2 (I suspect not).

[juru1234]

I understand your arguments. Maybe we can just add something to the neovim docs that developers should pay attention to use Lua 5.1 syntax when their plugin doesn't require luajit?

[clason]

No, nothing has changed. The fundamental reasoning is still the same and sound: we do not want to tie ourselves to a specific interpreter precisely to allow for choice. The fact that we use LuaJIT for our official releases is only because it's currently the best available interpreter for those platforms. If at some point we add official releases for s390x, we'll pick an interpreter to build against for it (very likely LuaJIT2, simply because it requires no build system changes -- but we'll burn that bridge when we come to it).

Note the "currently" here, which is load-bearing: if a better interpreter comes along (e.g., luajit-remake, if that ever becomes a ting), we'll switch in a heartbeat.

[clason]

Maybe we can just add something to the neovim docs that developers should pay attention to use Lua 5.1 syntax when their plugin doesn't require luajit?

As bfredl wrote, the documentation is already quite clear on that; if developers don't read that, I don't know what to do. Until the mythical "Plugin writing guide" happens, there's nothing actionable here.

[dundargoc]

Great call on the plugin guide, I think this is a good candidate to mention to raise awareness.

[dundargoc]

@juru1234 which distros are you using that builds with lua on s390x? Perhaps we can convince them to use luajit2 instead.

[juru1234]

@dundargoc The main distros are Ubuntu, Fedora, SLES and RHEL. This is kind of a chicken egg problem here. The distros want to ship as close as upstream so they ship with luajit. And they also want to ship with the same luajit version for all architectures. I would really appreciate if you help me to convince them to use luajit2 instead of luajit but then you would have the risk that they also ship neovim with luajit2 on x86.

[juru1234]

@dundargoc And yes, I think Fedora 40 and openSUSE currently ship neovim with luajit on s390x. But this is based on the patch that is ignored upstream and not based on luajit2. So this isn't a good long term solution. If the patch doesn't apply anymore, they will drop the support.

[clason]

Or switch to LuaJIT2. Predictions are hard, especially about the future. (To be clear, I consider patching LuaJIT the better approach over switching to LuaJIT2.)

I'm not sure what you're expecting of us here; if the distro maintainers have questions, they are welcome to ask us, but we will not change reasoned project policy based on downstream hypotheticals.

And if you're unhappy with distro packages, you can always build from source yourself; it's really not hard.

[juru1234]

@clason I don't expect anything here this was just an answer to @dundargoc's question.

[juru1234]

@clason And yes, building neovim with luajit2 isn't really hard but when you have a lot of test systems it gets a bit annoying when you have to do it on every system you need for work. This is why I decided to go for a config that doesn't depend on LuaJIT.

[dundargoc]

I think @jamessan packages neovim for ubuntu. Perhaps he has some insight whether packaging luajit2 instead of lua for s390x is feasible.

[jamessan]

This is kind of a chicken egg problem here. The distros want to ship as close as upstream so they ship with luajit. And they also want to ship with the same luajit version for all architectures. I would really appreciate if you help me to convince them to use luajit2 instead of luajit but then you would have the risk that they also ship neovim with luajit2 on x86.

This is all up to the distros. From Neovim's side, the only policy is "link against a Lua 5.1 implementation".

Since LuaJIT isn't available on all architectures, distros are already using different implementations on different architectures (LuaJIT where it's available and Lua where not).

As far as LuaJIT vs LuaJIT2, Debian actually had both for a while because there's some other software that requires LuaJIT2 (nginx, I think). A few months back, the package maintainers unified on the LuaJIT2 implementation but providing it under the LuaJIT name, so there wasn't a need (on the Debian side) to also differentiate between which LuaJIT implementation to use per architecture. However, we could have easily done that if necessary.

I can double check whether s390x is working with LuaJIT as part of the work to get 0.10 packaged.

[clason]

nginx

That is correct, as far as I know. And that sounds like a perfectly reasonable strategy; as far as Neovim is concerned, there is no need to distinguish either (any "compatible fork" will do). But note that LuaJIT2 provides further extensions on top of LuaJIT's extensions (on top of PUC Lua 5.1), and these are definitely not safe to use in plugins.

It would be good to confirm and document that, though; that could be an open door for confusion in some situations...

[juru1234]

Do I understand it correctly that Debian/Ubuntu will ship neovim with the luajit2 implementation on all architectures? @clason Isn't that reson enough to also switch upstream to luajit2? ;) + the behavior of the luajit maintainer

[clason]

@clason Isn't that reson enough to also switch upstream to luajit2

No, and stop asking.

[lewis6991]

Hi, we are in a situation where lua plugins use e.g. goto's in their code that isn't compatible with Lua 5.1 because 99% use neovim with LuaJIT that does support goto's. Because of this, these plugins cannot be used with neovim without LuaJIT even if they don't require LuaJIT at all.

If plugins use goto then they do require LuaJIT. An obvious solution is to just update these plugins to not use goto if that is all they are using from LuaJIT.

[dundargoc]

An obvious solution is to just update these plugins to not use goto if that is all they are using from LuaJIT.

I'll give the OP the benefit of the doubt that they're probably aware and that this was larger/long term solution they were looking for.

Anyway, I've confirmed debian, Fedora and RHEL all support luajit on s390x, either via using luajit2 or by patching luajit. So I believe this issue is resolved.