ChimeraPy / Engine

Distributed computing framework for Multimodal data written in Python
https://chimerapy-engine.readthedocs.io
GNU General Public License v3.0
8 stars 0 forks source link

ChimeraPy DAG should be dynamic vis-a-viz data sources. #294

Open umesh-timalsina opened 10 months ago

umesh-timalsina commented 10 months ago

Scenario

Let's take a client application. What are the realistic possibilities that a ChimeraPy pipeline is in sync with the operations in the client applications? While this is an ideal scenario for every client application to write ChimeraPy node and orchestrate a pipeline every time a new session happens, we can do one better by going a level lower. This is not a "time-aligned" recording, but rather a dynamic datachunk that can be fed into the processing graph based on some sort of auth mechanism. Such that nodes can process data based on what is fed to them. And with async processing functions, we can definitely do this even if the from different sources arrive at different timepoints in the cluster.

Applications

While we have been delving into what is the best way to integrate ChimeraPy with learning environments, or any other application for that matter? This approach is definitely more flexible; can support language agnostic API/ protocols and I can see the following applications for this: