desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
36 stars 24 forks source link

consolidate psf and flat jobs #2371

Open sbailey opened 2 months ago

sbailey commented 2 months ago

One of the challenges of running kibo was the sheer number of jobs, ~50k. This required extra queue management to avoid submit limits, made rare startup problems common to encounter, and added overhead to reservations which sometimes had ~10% of the nodes idle while turning over jobs. To reduce the number of jobs,

Although there are still more science jobs than there are arc/psf/flat/nightlyflat jobs, these calib jobs are tightly coupled and blocking factors for getting to the more genuinely independent tilenight and ztile jobs, so bundling them would streamline the workflow.