We want to introduce data stores on OpenFn to allow users to store, preview, and reuse data across projects, workflows, and steps.
We want to be able to refer to these data stores as collections in the steps e.g. a “save to collection” step in a workflow will store the parsed data into a collection given a path.
E.g., workflow one receives data from X in real time, converts to “patients” in some standard (fhir?), and persists it to /project/collections/patients. Workflow 2 then gathers data from the data collection every day, aggregates it, and sends it to dhis2.
Maybe two step types: pull from collection and push to collection.
Users should be able to preview their collection and also query the data collection via a REST API endpoint.
We want to introduce data stores on OpenFn to allow users to store, preview, and reuse data across projects, workflows, and steps.
We want to be able to refer to these data stores as collections in the steps e.g. a “save to collection” step in a workflow will store the parsed data into a collection given a path.
E.g., workflow one receives data from X in real time, converts to “patients” in some standard (fhir?), and persists it to /project/collections/patients. Workflow 2 then gathers data from the data collection every day, aggregates it, and sends it to dhis2.
Maybe two step types: pull from collection and push to collection.
Users should be able to preview their collection and also query the data collection via a REST API endpoint.