zincware / ZnTrack

Create, visualize, run & benchmark DVC pipelines in Python & Jupyter notebooks.
https://zntrack.readthedocs.io
Apache License 2.0
44 stars 4 forks source link

dynamic graph writing #770

Open PythonFZ opened 5 months ago

PythonFZ commented 5 months ago

do not run the graph using dvc repro but some zntrack repro <script.py> which write the node only after/ shortly before it is being executed. Once you encounter an if node.something you check if the nodes afterwards exist? Otherwise, you load the node and use it for iterating. This should include parallel execution.