search

numaproj / numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs
https://numaflow.numaproj.io
Apache License 2.0
1.3k stars 116 forks source link

structured concurrency is lost in actor pattern #2151

Open vigith opened 1 month ago

vigith commented 1 month ago

spawn handlers are lost in https://github.com/numaproj/numaflow/blob/main/rust/numaflow-core/src/source.rs#L111-L126

vigith commented 1 month ago

the moment we add handler to struct, Clone will no longer be applicable

BulkBeing commented 1 month ago

I think we should rely on message passing to shutdown tasks. Another possible option is to return join handles in a wrapper struct and abort tasks within the Drop implementation of it.