Closed sertel closed 5 years ago
So this means we retire dataflow operators entirely? (I'm not against it, it kinda complicates the API)
No, the other way around. We retire DataflowFunctions entirely.
(My hope is actually, that we can generate the code for the required operator eventually.)
In our (current) JVM implementation, we allow the compiler author to write more powerful functions. These mechanisms are as powerful as writing a dataflow operator itself but do not require dataflow abstractions such input ports or output ports. Instead they rely on the concept of non-strict functions realized via continuations. As a result, we currently flag
ohua.lang/smapFun
andohua.lang/oneToN
as stateful functions instead of operators and therefore force the backends to provide these non-strict functions.However, many languages do not have the concept of continuations and we realized in the JVM version that providing this feature complicates the code a lot.
As such, we should not force the backend to have this functionality and flag
ohua.lang/smapFun
andohua.lang/oneToN
as normal dataflow operators.