search

terralang / terra

Terra is a low-level system programming language that is embedded in and meta-programmed by the Lua programming language.
terralang.org
Other
2.72k stars 201 forks source link

Moonjit? #438

Closed runningskull closed 2 years ago

runningskull commented 4 years ago

Moonjit is an active fork of LuaJIT that’s implementing features from newer lua versions while maintaining full backward compatibility with LuaJIT. They also intend to maintain ppc support so it might be a way to keep LuaJIT’s speed and ffi (per https://github.com/terralang/std/issues/4) and avoid having to switch to PUC like in #320

How feasible would it be to use it in Terra? From my quick poking around, it seems possible that it’s fairly simple. I’ll give it a proper attempt when I have time, but maybe someone else will have a better sense for whether it’s worth pursuing.

elliottslaughter commented 4 years ago

First off, I'll just say in general I'm supportive of all these LuaJIT forks. I hadn't heard of Moonjit specifically before but if it continues to pick up momentum, it may eventually become more viable than LuaJIT.

I don't think it'll remove the need for PUC Lua, if I understand what that issue is saying, because the issue is talking about PPC (i.e. 32-bit) support, which is (or was) pre-existing in LuaJIT. Whereas for Summit (and presumably any future PPC machine) we need PPC64le. This is something for which there are forks, but every one I've ever tried has been buggy. It has never existing in mainline LuaJIT.

Having said that, if it's build-compatible with LuaJIT it should be easy to test out. We already have variables that configure the URL, version, etc. of the LuaJIT build. If we need another couple of variables (e.g. in case they renamed the binaries and libs) then I'm happy to take patches to support that. I don't think I'd change their default values (yet), but I'm happy for it to be a build-time option.

runningskull commented 4 years ago

Agreed - here's hoping one of the forks carries the torch. Thanks for the clarification on my ppc mixup. I'll update this thread if I get anything interesting.

Terra is a very cool project btw. Thanks for the good ideas and work that have gone into it.

Xaekai commented 4 years ago

I'm surprised you hadn't heard of moonjit, since it's the fork of the author of the rather well-known (at least among those who have vested interest in the future of Lua) article/blog post "A JIT in time..."

https://siddhesh.in/posts/a-jit-in-time.html

elliottslaughter commented 4 years ago

I just noticed that Moonjit claims to support PPC64le. I'll try to check this out and see if it works for my use case. The previous forks I tried were all horribly broken, so I'm cautious until I can test out some of my larger Regent codes to be sure this actually runs. But if it works then we could avoid the need to actually merge PUC Lua support and its associated hacks.

Edit: in case you want to follow the discussion I'm having with the Moonjit maintainers: https://github.com/moonjit/moonjit/issues/128

elliottslaughter commented 4 years ago

Ok, there's an initial patch here: https://github.com/terralang/terra/pull/452. It's breaking somewhere in the bytecode compilation for the internal Lua files. Not sure what's going on, maybe I'm not building it correctly or something.

elliottslaughter commented 4 years ago

Moonjit support is in master and is enabled by default. You can select the Lua implementation with TERRA_LUA (e.g. TERRA_LUA=luajit to switch back to LuaJIT). I'll continue to watch the various forks over time to see how they do, but at least for the moment this allows us to support PPC64le, so it's clearly a win for users who need to run on Summit.

elliottslaughter commented 4 years ago

We may need to revisit this decision as Moonjit is no longer being actively maintained. This may not actually be worse than LuaJIT in practice, but at any rate we'll want to keep an eye on both to see if either project picks up a new maintainer.

Moonjit still provides valuable support for PowerPC, so I don't expect any practical changes any time soon, but in the longer term we may need to revive PUC support (again).

https://github.com/moonjit/moonjit/commit/a2a39ea7184f3c8cab9474c6e41f6541265fb362

norcalli commented 2 years ago

LuaJIT is being actively maintained by Mike, so it seems better in that regard (but still no PowerPC). Out of curiousity, what PowerPC machines are people using? The only examples I can think of that might still be common are game consoles.

elliottslaughter commented 2 years ago

IBM is active in high performance computing (HPC). So any machines they build will generally be PowerPC. E.g., Summit (https://www.olcf.ornl.gov/summit/), which was number 1 in the world when it first came out. I believe there are also some machines in Europe with the same architecture.

Once Summit (and its generation of sibling machines) is decommissioned, my personal interest in PowerPC will shrink substantially. However, it is always possible that IBM will win a future contract with one or more supercomputing facilities and we'll see a future generation of Power machines show up again.

I am glad that Mike is continuing to maintain upstream LuaJIT. I still think that longevity may be an eventual issue for this project though. While I don't follow that closely, the last I checked Mike had shot down all community attempts to build an official test suite, which makes it challenging for third parties to contribute to the project when they can't test their own changes. For that matter, we don't even necessarily have confidence that Mike's own changes are bug-free, as I understand that he only really ran the full (private) test suite on each release, and we still haven't seen one of those in almost 5 years.

elliottslaughter commented 2 years ago

In the time since my last comment, Moonjit was formally archived and made read-only. This doesn't really change the situation, just confirms what we already knew.

In addition, we found a LuaJIT bug that was fixed upstream but not in Moonjit. Therefore, I merged #568 to make all non-PPC64le platforms use LuaJIT (again) by default.

I'm going to close this as the Moonjit saga seems to be over. The archived version seems to serve our PPC64le userbase sufficiently well; as best I know, there aren't any Moonjit bugs specifically holding those efforts back. If there were, I'm not sure what we'd do, but at any rate, Moonjit does not seem to be the future.

I appreciate everyone's time and effort to comment on this issue.