Support Multithreading in Lua

Limit-Theory-Redux / ltheory

Limit Theory Redux is a fork of the discontinued open-world space simulation game Limit Theory. We have made it our mission to continue the development of the ambitious Limit Theory project as an open source initiative.

Apache License 2.0

62 stars 8 forks source link

Support Multithreading in Lua #102

Open dgavedissian opened 1 year ago

dgavedissian commented 1 year ago

Currently, the Lua VM is completely single threaded with some cooperative multitasking support using coroutines. This will still saturate the main thread and limit performance.

Some multithreading frameworks exist (with significant code support on the native side), such as:

I think the rough idea is that we want to have one Lua VM for each thread, and utilise some kind of message passing protocol to communicate between the "threads". That means passing only basic Lua objects like strings/ints/etc that can be serialized/deserialized to bytes, essentially equivalent to a networking protocol but within a single process.

IllustrisJack commented 1 year ago

Lua problems:

Heavy tasks like iterating large tables or running functions that need to do so e.g. system creation will bring the main thread pretty much to a halt. This will make the game almost crash as it depends on the Lua VM to handle everything. Especially system & ship generation freeze the main thread until it's fully done and we can't simply pause / the user can't do anything else in the application while these are happening.
Also I can see things like the economy simulation etc being offloaded as these put unnecessary strain on the main thread.

IllustrisJack commented 1 year ago

Currently a system is generated like this https://github.com/Limit-Theory-Redux/ltheory/blob/main/script/GameObjects/Entities/Test/System.lua

While there are lots of things that can still be optimized in lua (already on it) we will eventually reach a ceiling which needs to be addressed sooner or later.