Closed GeigerJ2 closed 1 month ago
The graph is almost sufficient but not quite. There is info we need from the runtime section, typically if input data already exists.
That is outdated now and done through parsing/unrolling
Just for context: This is an older issue from the time we called cycles
in the yaml file graph
.
From my understanding, the "graph" section under "scheduling" is fully sufficient to define the graph, as it defines the tasks, their frequency, inputs, outputs, dependencies, etc. This is in a rather abstract manner, and tasks and data nodes are being made explicit by providing paths to executables and files / folders in the "runtime" section. However, these are, in principle, not needed to define the abstract graph initially. Therefore, I propose to enable graph creation purely from the "graph" section.
Like this, users can create and iterate on the workflow they want to run in an abstract manner, without even having to think about any runtime parameters or where scripts / files will eventually be located.