Open IS4Code opened 1 year ago
Full ack.
I started this project in the year 2012. And async/await didnot exists. That was the reason why I used threads in the first place. To implement coroutine.yield
. I am also not happy with the solution. That is the reason, why but a simple async/await implemention on top in our "company" projects (this is partly open in the DES-project). But this is only a bad fix for it.
To support real async/await/cooperative multitasking, there is a need of a state machine. To keep it fast. A first problem is the syntax tree, that I avoided in the current implementation. This tree makes the split in the state parts easier. Because you need to keep track of the code flow and lifted local variables. If this will sometimes implemented, the rest will be easy with an own implemention of SynchronizationContext
.
Bad is, there is no help within the Linq Framework (as I know). So, the complex logic that the c# team did, must be redone for NeoLua. Or do you know some library?
I am afraid I don't know of a usable library for that, but I wish you good luck on solving this!
I am interested in using NeoLua as a scripting language for a game. For this I will need to have many coroutines running at the same time that need to be pretty lightweight. I am talking about a few hundred, maybe event thousands at the same time.
Is this possible with the current implementation? If not, are you considering improving the implementation like @IS4Code suggested?
I really hope this works out, because I also tested other lua implementations for .net and even other scripting languages and NeoLua was way faster than all others.
I need to start thousands or even 10K coroutines Do you have any plans to make changes? @neolithos
I am interested in using NeoLua as a scripting language for a game. For this I will need to have many coroutines running at the same time that need to be pretty lightweight. I am talking about a few hundred, maybe event thousands at the same time.
Is this possible with the current implementation? If not, are you considering improving the implementation like @IS4Code suggested?
I really hope this works out, because I also tested other lua implementations for .net and even other scripting languages and NeoLua was way faster than all others.
I have the same needs as you, but neolua cannot be used in our project until the coroutine changes.
I have a lot open issues in my payed job. So, this will take a while... Hopefully, over x-mas I will have a little bit free time.
I have already tested small parts, but I need to rewrite the whole parser and syntax tree.
Is there a nice async/await framework for linq somewhere?
This is an amazing project from what I've seen, especially the DLR interaction, but I am a bit concerned by the
coroutine
implementation.As a target introduction, coroutines in Lua should essentially allow you to effortlessly do something like this:
The core part is that in C# you would have to use
await
to continue thehttp.fetch
call with code that operates on the result of the task, but as a consequence you have to mark every method asasync
. Lua does not have this limitation ‒ if you have a global scheduler that can spawn coroutines as needed, a single thread can switch between execution of active coroutines and implement I/O, waiting etc. using simple, non-blocking functions. This is why Lua is amazing for embedded applications and games or other event-based environments especially.The point is that Lua coroutines are not (CPU/OS) threads, they are fibers and they use cooperative multitasking. This is why they should be amazing for interaction with TPL in .NET, and you could actually convert between coroutines and tasks with the help of a scheduler (a coroutine can have its
TaskCompletionSource<LuaResult>
and you can callcoroutine.resume
throughTask.ContinueWith
). They are also a generalized form of iterators, so a coroutine should also be anIEnumerator<LuaResult>
, and callingMoveNext
should callcoroutine.resume
.My issue here (and please correct me if I am wrong) is that based on the
LuaThread
implementation, you seem to be creating actual .NET threads (throughTask.Factory.StartNew
) which obviously take resources, and based on your recommendation of using.Wait()
to call async methods, you are blocking those threads from doing anything useful. This is terrible! Theasync
/await
APIs exist in the first place to move away from blocking to event-based API while keeping the convenient usage without cumbersome continuation-passing and closures. It's true that you can implement thecoroutine
API using threads and you can mimic fibers using threads, but that leaves you with all the heaviness of threads but with less capability.I suggest writing a completely new code that actually does use state machines to implement this API, but I understand it may not be easy ‒ every call to a function, every indexing, every operator, and even every instruction (if hooked by
debug.sethook
) in Lua may yield, and I assume it would be very hard to compile that to IL; in essence turn every code into a state machine similarly to what iterators in C# do. However, making this would make it possible to have thousands of coroutines without depleting system resources, which would work even in single-threaded environments (WebAssembly), and interact with other systems (TPL or Unity's coroutine), which is definitely worth the effort.