Parallel Processing for Masters

Keck-DataReductionPipelines / KPF-Pipeline

KPF-Pipeline

https://kpf-pipeline.readthedocs.io/en/latest/

Other

11 stars 2 forks source link

Parallel Processing for Masters #917

Closed bjfultn closed 3 months ago

bjfultn commented 3 months ago

use --watch, --reprocess, and --masters with --ncpus to process masters in parallel
processing masters in --date mode is broken (L1 objects not updating properly in the loop) for an unknown reason and this is a workaround that fixes that problem

bjfultn commented 3 months ago

@RussLaher please review and test this before we merge. I changed the call for the L1 part of the masters DRP to include these new flags and it should operate in parallel.