valohai / valohai-utils

Python helper library for Valohai
MIT License
2 stars 2 forks source link

Building a pipeline in Python (valohai-utils) #26

Closed skogstrom closed 3 years ago

drazendee commented 3 years ago

This popped up today with a VH user as one of the things they hope valohai-utils could do.

They currently have a pipeline that has 10 nodes and then each of those nodes goes to 10 other nodes (so 100 parallel nodes). Writing that by hand is a pain in the ass, so they created a script to generate the node definitions and edges. Would be great if valohai-utils would support that, so folks wouldn't have to write their own scripts.