desihub / desispec

DESI spectral pipeline
BSD 3-Clause "New" or "Revised" License
33 stars 24 forks source link

afterburners register/launcher framework #1290

Open julienguy opened 3 years ago

julienguy commented 3 years ago

We could add to the pipeline the framework to run afterburners

This would be similar to what we developed for nightwatch QA, see https://github.com/desihub/nightwatch/blob/master/py/nightwatch/qa/base.py

We would write in this framework wrapper classes for quasarnet , the Mg line fitter for QSOs, the [OII] doublet line fit for ELGs ...

sbailey commented 3 years ago

Another thing to consider is defining parallelism, memory, and runtime needs. We had a hard enough time sorting that out with desispec.pipeline.tasks.base.BaseTask with pipeline pieces where we had full control; it could get messy with end-user contributed afterburners.