do not run the graph using dvc repro but some zntrack repro <script.py> which write the node only after/ shortly before it is being executed. Once you encounter an if node.something you check if the nodes afterwards exist? Otherwise, you load the node and use it for iterating. This should include parallel execution.
do not run the graph using
dvc repro
but somezntrack repro <script.py>
which write the node only after/ shortly before it is being executed. Once you encounter anif node.something
you check if the nodes afterwards exist? Otherwise, you load the node and use it for iterating. This should include parallel execution.