Copy in-progress greenlets

[Gurkenglas]

def foo(bar): 
    print(1)
    bar()
    print(3)

def main(): 
    c = baz(foo)
    print(2)
    c()
    c()

I'd like to be able to implement baz such that main prints 1,2,3,3. It seems like your library could support this.

[jamadden]

Your description is unclear and seems buggy (what is bar? Why does baz(foo) not pass bar to the foo function? Why do all these generic names that do not help describe the code?), but it seems like you want to be able to capture the continuation of a greenlet at some point and then return to it, separately, as many times as desired?

I think greenlet is not designed to support that, even if some of the required infrastructure is there. There would have to be a really great use-case to motivate the required work, which would likely involve touching a lot of platform specific assembly.

[Gurkenglas]

bar is the callback argument of foo. baz designs a bar to pass to foo.

A baz that doesn't support copying would be:

def baz(foo):
  ret = None
  gfoo = greenlet(lambda: foo(lambda: ret.switch()))
  def c():
    nonlocal ret
    ret = greenlet.current()
    gfoo.switch()
  c()
  return c

The predicted implementation would be that copying a greenlet would copy its stack (the portion belonging to itself, not its parent), perhaps defering the copying of each frame until that frame would be touched, if such is supported.

The use case would be nondeterministic computation: If an immutable list provides an implementation of map, one could use greenlets to combine (1,2) and lambda x: [x, x+100] into [(1,2),(1,102),(101,2),(101,102)].

Specifically, I am using greenlets to turn a python-lenses Setter into a Traversal in a single pass, and this change would allow that Traversal to support more Applicatives.

Mathematically, the greenlet library should be describable as providing a function ((A->B)->C) -> Union[C,(A,B->((A->B)->C)]. That is, run a function until it calls a callback, then return its current state. It does this except that this state can only be consumed up to once.

[jamadden]

bar is the callback argument of foo. baz designs a bar to pass to foo.

The names are still trivially short and don't help convey the intent of the argument...

Arguments from mathematical purity aside, attempting to have reasonable semantics for copying and re-entering arbitrary mixed C and Python stacks is a hard problem to solve (especially if you expect reasonable time complexity, and if you consider the interaction with the rest of the highly-mutable runtime system; for example, I see no way to avoid deeply copying and rewriting the entire Python stack of a "copied" greenlet, as on-demand copying isn't something greenlet can manage — it isn't part of the evaluation loop and can't know when a frame is touched).

As currently described, I believe such a use case to be outside the scope of greenlet. greenlets are primarily designed to provide cooperative multitasking, and most commonly are used with IO frameworks like gevent or eventlet; their primary purpose is not to be functors and most people would consider them heavyweight in that application.

Similar special-purpose implementations I'm aware of that do allow (some variation of) reentering arbitrary stacks require deep support in the lowest levels and use techniques like software transactional memory, memory fences, other unportable means, or indeed, an entirely different runtime system. But most of those special-purpose implementations are targeted at debugging, when it comes to C, Python, and similar language families. Languages I know of that that deliberately support continuation capture and reuse, like Scheme and certain Smalltalk dialects, have that designed into the language/compiler/runtime system — and I'm not sure they support mixing that feature with their FFI; I can't find any information on that. greenlet is not built into the language, and mixing well with the FFI is a key feature.