group_by is one of the few operators that spawns AsyncIterators rather than reducing them. A map-style lookup of keys is also central to any reasonable implementation, and the two combined spell Subject. This is because for efficiency sake only one scope should be responsible for reading off the original Producer and performing the key lookup, but any iterator could receive the value.
An aside on Subject
HHReactor purposefully doesn't have a dedicated Subject — partly because generalized Subject requires an infinite Awaitable that doesn't eat cpu cycles, which is impossible in Hack right now; partly because timing and soundness is hard enough when all the machinery is confined to private variables in Producer, and I fear giving control to the outside world might set something on fire.
That said, the unidirectional flow of data from Producer to consumer assumes the Producer is at least as qualified to know when and what to spawn and emit as the consumer is to request them. However, the asymmetry of Maps gives the consuming scope more power than the producing scopes which, since they can't communicate, can only search linearly by doing a checking personally and passing the control along.
group_by and Subject
The compromise is a private Subject confined to generalized_group_by using a Map of ConditionWaitHandles. However, some of the timing is dubious, notably the creation of these makeshift Subjects which can't be executed eagerly, or else the ConditionWaitHandle will throw. Furthermore, since these Subjects aren't cloned, the clone-safe HHReactor\Queue has been swapped for SplQueue for simplicity, but I haven't yet scrutinized the decision.
group_by
is one of the few operators that spawnsAsyncIterator
s rather than reducing them. A map-style lookup of keys is also central to any reasonable implementation, and the two combined spellSubject
. This is because for efficiency sake only one scope should be responsible for reading off the originalProducer
and performing the key lookup, but any iterator could receive the value.An aside on
Subject
HHReactor
purposefully doesn't have a dedicatedSubject
— partly because generalizedSubject
requires an infiniteAwaitable
that doesn't eat cpu cycles, which is impossible in Hack right now; partly because timing and soundness is hard enough when all the machinery is confined to private variables inProducer
, and I fear giving control to the outside world might set something on fire.That said, the unidirectional flow of data from
Producer
to consumer assumes theProducer
is at least as qualified to know when and what to spawn and emit as the consumer is to request them. However, the asymmetry ofMap
s gives the consuming scope more power than the producing scopes which, since they can't communicate, can only search linearly by doing a checking personally and passing the control along.group_by
andSubject
The compromise is a private
Subject
confined togeneralized_group_by
using aMap
ofConditionWaitHandle
s. However, some of the timing is dubious, notably the creation of these makeshiftSubject
s which can't be executed eagerly, or else theConditionWaitHandle
will throw. Furthermore, since theseSubject
s aren't cloned, the clone-safeHHReactor\Queue
has been swapped forSplQueue
for simplicity, but I haven't yet scrutinized the decision.