Currently, we use a separate SourceTasks that runs these in Tokio. It may be easier to manage if we had a separate SourcePipeline although that raises questions of how a pipeline interacts with Tokio. Deferred for now since the "tokio pushes directly to the first non-SourcePipeline" is working.
Currently, we use a separate
SourceTasksthat runs these in Tokio. It may be easier to manage if we had a separateSourcePipelinealthough that raises questions of how a pipeline interacts with Tokio. Deferred for now since the "tokio pushes directly to the first non-SourcePipeline" is working.