[Enhancement] Coroutine

[animehunter]

Would be nice to add coroutine to the language. It could be used to implement useful things like generators, async/await, state machines, actors, etc

[Skrylar]

Does the trick of abusing an IEnumerable and manually calling next on it work in Beef? That's how its janked together in C#.

[bfiete]

No, there's no "yield return".

[MaddyThorson]

I just want to add a +1 here!

yield return is really the only important feature missing for us, coming from C# game development. It greatly simplifies gameplay/cutscene timing logic. If you would like info about our use case, let me know.

[bfiete]

This C# feature leans heavily on the C# GC- internally the method is broken into a state machine and a local state capture is GC-allocated to pass local variables and the state index back into the enumerator (which makes those local variables not actually local variables anymore). C# disallows dynamic stackalloc expressions in iterators to avoid the issue of a side-stack, so all allocations in that method already occur on the heap so those just get cleaned up by the GC eventually anyway.

For Beef, making this stuff work is pretty far along the road to a full async/await system. We have the added complications of having to manually manage a fake "stack" of memory for the state machine for our full stack-allocation support... and there'd have to be some good thought put into how these variably-sized local state captures would be allocated and freed while supporting custom allocators...

It's actually a pretty complicated feature. I wonder if there's some simpler partial solution to your timing logic problem...

For those who don't know what I mean by the state machine, here's one article that explains it: https://startbigthinksmall.wordpress.com/2008/06/09/behind-the-scenes-of-the-c-yield-keyword/

[MaddyThorson]

Ah, thanks for the explanation. We'll try to implement something as close to coroutines in Beef as we can as it stands, and maybe that will help us find suggestions more appropriate than yield return.

[bfiete]

@MattThorson Maybe something like a sequencing system based on a List of delegates processed one at a time? Those delegates could be lambdas or local functions with captures... you could treat the delegates as either continuations (adding the next delegate after the current one is executed) or populate the list all at once. Maybe there's something ergonomic enough there...

[isral]

@bfiete You don't need GC. Check on my failed proposal serialization result: https://github.com/dotnet/csharplang/issues/2741

The implementation is C duff device with local variable become object fields. I try it in C++ and works (but without serialization). The call stack is just array of coroutine (works on c# too).

I try to create my own language because I can't find language with serializable coroutine but I don't make much progress.

[Skrylar]

Lua’s coroutines can be used to make an entire async/await system, among other things.

Things do get a little strange if a coroutine creates more coroutines if I recall.

[kochol]

Fibers are great to create coroutines because you can pause them at any time and continue them when you want. Switching them is cheap as setting one pointer.

But as far as I know, it has problems with Beef memory leak detector as it changes the stack.

look at sx library https://github.com/septag/sx/blob/47d3162d6a3cf214ddf33ab069f41e773fc8d2d0/include/sx/fiber.h#L57

[xposure]

Thinking a little outside of the box from when I wrote a light weight threading system on z80, couldn't you just do some fake threading? Its not so much about yield than we want to give up our current execution time and let something else continue and then come back and resume.

At the lowest level, this just requires saving the registers and stack pointer and then on your other microthread you would just move back to its stack and restore its cpu registers. I know cpus are far more advanced that when I messed with this stuff, but just trying to add a different perspective to the problem.

So my question is, could we implement a coroutine system that is a extremely light weight threaded system (no context switches) that is just saving the stack pointer and registers and jumping around between these co routines?

[Shadowblitz16]

I like the idea of using for loops to wait for things aka pausing..

bool SomeExternalState = false;
int MyIntCoroutine()
{
    SomeExternalState = true;
    while(SomeExternalState)
    {
        yeild()
    }
    return 0;
}

void Call()
{
    yield MyIntCoroutine();
}

The issue is I think the sync keyword shouldn't be a thing as it requires more typing for everything and has to be used all the way from main. (aka I think all functions and methods should be async)