this PR is a pretty big refactor of how we run lua code on crow. previously we would save all the baked-in lua files in their regular string representation & load (compile) them at runtime on crow.
this PR includes an additional luacc (cross-compilation) stage which compiles the lua files into lua bytecode and saves that into the flash image. the benefit is much reduced runtime RAM when loading libraries as they can be loaded directly into the VM without needing to be parsed at runtime.
the RAM used by the lua machine is not decreased, nor is FLASH space decreased (bytecode is similar in size to the strings), but it's just the compilation process that doesn't need to allocate anything.
while this has improved the situation where crow would freeze when running scripts that utilize multiple different ii devices, it still hasn't solved the problem. previously we would fail on the 2nd ii device being loaded, and now we fail on the 3rd. it's also not a matter of the garbage collector needing to be tuned, as a full GC cycle is run after any file is loaded into the VM.
still, this feels like an improvement of the system, and i would like to push it into main. that said, there's no point rushing it out now as we still haven't solved the fundamental problem which led me to work on this.
and the binary luacc is created at src/luacc.
it is just copied into this repo for convenience (and bc i can't have 2 submodules referencing the same repo afaik).
it should be easily converted for other lua versions should we need. just need to make sure to pass -DLUA_32BITS to the compiler. the only file changed is src/ldump.c in DumpHeader where we override the sizeof size_t and int to be 32 bits (uint32_t).
//
unfortunately the github action is broken because apparently luacc needs to be compiled for whatever system they are running on which i have no real sense of. it should be as easy as changing make linux to make <platform>, but we'd need to run that compilation in the action which i have no clue how to do.
tehn
commented
1 year ago
this PR is a pretty big refactor of how we run lua code on crow. previously we would save all the baked-in lua files in their regular string representation & load (compile) them at runtime on crow.
this PR includes an additional luacc (cross-compilation) stage which compiles the lua files into lua bytecode and saves that into the flash image. the benefit is much reduced runtime RAM when loading libraries as they can be loaded directly into the VM without needing to be parsed at runtime.
the RAM used by the lua machine is not decreased, nor is FLASH space decreased (bytecode is similar in size to the strings), but it's just the compilation process that doesn't need to allocate anything.
while this has improved the situation where crow would freeze when running scripts that utilize multiple different
iidevices, it still hasn't solved the problem. previously we would fail on the 2ndiidevice being loaded, and now we fail on the 3rd. it's also not a matter of the garbage collector needing to be tuned, as a full GC cycle is run after any file is loaded into the VM.still, this feels like an improvement of the system, and i would like to push it into
main. that said, there's no point rushing it out now as we still haven't solved the fundamental problem which led me to work on this.//
note the cross-compiler is built from this branch https://github.com/trentgill/lua/tree/luacc-5.3.4
invoke it from the root directory with:
and the binary
luaccis created atsrc/luacc. it is just copied into this repo for convenience (and bc i can't have 2 submodules referencing the same repo afaik).it should be easily converted for other lua versions should we need. just need to make sure to pass
-DLUA_32BITSto the compiler. the only file changed issrc/ldump.cinDumpHeaderwhere we override the sizeofsize_tandintto be 32 bits (uint32_t).//
unfortunately the github action is broken because apparently
luaccneeds to be compiled for whatever system they are running on which i have no real sense of. it should be as easy as changingmake linuxtomake <platform>, but we'd need to run that compilation in the action which i have no clue how to do.