search

polymec / polymec-dev

The development "branch" of the polymec HPC libraries.
Mozilla Public License 2.0
2 stars 2 forks source link

Explore some very basic JIT stuff using Lua -> libjit #342

Closed pbtoast closed 5 years ago

pbtoast commented 5 years ago

It would be neat if we could define various functions in Lua and have them compiled to (relatively) high-performance components.

LLVM is the compiler construction kit that all the kids use these days, but libjit seems to be simpler and possibly faster at compiling code. Libjit uses the LGPL license, so we'd have to load it as a shared library, which would be a performance hit, but it'd still probably outperform native Lua functions by a fairly wide margin. Especially if we provide a multi-eval interface in addition to the usual single-eval one for functions. In any case, it's easy to test the performance once it's working!

Approach:

  1. Initialize the LibJIT environment and open a new program.
  2. Compile a given string to a Lua chunk.
  3. For each statement in the chunk: a. Extract opcodes, constants, variables b. Translate them to their LibJIT equivalents.
  4. Close the program and get a handle to an executable LibJit Thing.

For details on how Lua's bytecodes work:

  1. See the Proto struct in lua/src/lobject.h, which represents a chunk. To get a Proto object from a compiled chunk, we can use the getproto() function mentioned in lua/src/luac.c.
  2. Check out opcodes and bytecode structure here: https://the-ravi-programming-language.readthedocs.io/en/latest/lua_bytecode_reference.html
  3. Here's an involved example of how one uses libjit's simple assembler language: https://thosakwe.com/building-an-esolang-jit-with-gnu-libjit/
  4. See also: https://eli.thegreenplace.net/2013/10/17/getting-started-with-libjit-part-1
pbtoast commented 5 years ago

Still to do:

  1. Figure out a strategy for storing the following in registers:

    • Functions (C/Lua)
    • Tables (stored in registry with integer refs, but no way to distinguish)
    • Strings (have to store length and data, or NULL terminate. Currently can't distinguish between these and other pointer types).

  2. Finish implementing opcodes: OP_SELF, OP_LEN, OP_CONCAT, OP_FORLOOP, OP_FORPREP, OP_TFORLOOP, OP_TFORCALL, OP_SETLIST, OP_CLOSURE, OP_VARARG, OP_EXTRAARG

  3. Debug and plug leaks.

pbtoast commented 5 years ago

This project is taking on enough of its own life that I think it might be best in a separate repo as an add-on to polymec.