search

isi-vista / adam

Abduction to Demonstrate an Articulate Machine
MIT License
11 stars 3 forks source link

Pipeline script #1188

Closed spigo900 closed 1 year ago

spigo900 commented 1 year ago

Implements a script to queue all preprocessing steps from one script. The preprocessing steps are queued as separate Slurm jobs with dependencies on each other, set up to cancel immediately if one of their dependencies is canceled/fails. This script can also supports running each step iteratively and interactively by running the Slurm scripts in Bash.

I haven't yet tested this with color segmentation or instance segmentation. Once I have both working I may add segmentation server startup to this.

spigo900 commented 1 year ago

I've since tested this with color segmentation and it works as far as I could tell. I did some work here to separate the segmentation and refinement scripts as well which means color and instance segmentation can run in parallel.

As for instance segmentation, I've added code for server startup and shutdown but still need to test it to confirm it works.

spigo900 commented 1 year ago

Testing is still in progress and should be done today assuming no more bugs to fix. Since the basic setup seems to work, I think this is ready for review now -- though I may push a few bug fixes between today and Monday.

spigo900 commented 1 year ago

Update: The pipeline finished, and as far as I can tell, everything works.