As documented in Ticket #134, there are 83 files in mtpipeline_outputs known to cause the pipeline to fail. 80 of these have an EXPTIME flag of 0, which causes AstroDrizzle to fail, and 3 others have an unknown problem
In Ticket #134, we looked at these last three images and determined they were of little value. The value of the other 80 is harder to determine, although having an EXPTIME flag of 0 likely indicates they are bad observations.
One way to deal with these files in the pipeline is to have something that checks the proposalid (like j9fw33r3q) against this list, and to abort after the cosmic-ray rejection step if it finds it. That's effectively what the pipeline already does, but this way we can throw in an error message into the log that this is a known bad file.
I'll create a 'bad-files' branch to add this change.
As documented in Ticket #134, there are 83 files in
mtpipeline_outputsknown to cause the pipeline to fail. 80 of these have anEXPTIMEflag of 0, which causes AstroDrizzle to fail, and 3 others have an unknown problemThe
EXPTIMEflag problemwfc3/11573_uranus/ib3805v2q_flt.fits wfc3/11573_uranus/ib3802tpq_flt.fits wfc3/11573_uranus/ib3802ttq_flt.fits wfc3/11573_uranus/ib3805v1q_flt.fits wfc3/11573_uranus/ib3802tsq_flt.fits wfc3/12237_174567/ibjb04p0q_flt.fits wfc3/12237_174567/ibjb04p1q_flt.fits wfc3/12237_2005ef298/ibjb07jdq_flt.fits wfc3/12237_2005ef298/ibjb07jcq_flt.fits wfc3/12237_2005ef298/ibjb07jbq_flt.fits wfc3/12468_04vy130/ibtp14dxq_flt.fits wfc3/12468_04vy130/ibtp14dyq_flt.fits wfc3/12468_04vy130/ibtp14e2q_flt.fits wfc3/12468_04vy130/ibtp14e0q_flt.fits wfc3/12468_08so266/ibtp22weq_flt.fits wfc3/12468_08so266/ibtp22wgq_flt.fits wfc3/12468_08so266/ibtp22wdq_flt.fits wfc3/12468_08so266/ibtp22wiq_flt.fits acs/11970_titan-pre-inf-conj/jb9z04dbq_flt.fits acs/11970_titan-pre-inf-conj/jb9z04dcq_flt.fits acs/11970_titan-pre-inf-conj/jb9z04d6q_flt.fits acs/11970_titan-pre-inf-conj/jb9z04d7q_flt.fits acs/11970_titan-pre-inf-conj/jb9z04daq_flt.fits acs/11970_titan-pre-inf-conj/jb9z04d8q_flt.fits acs/10805_uranus/j9q301g4q_flt.fits acs/10805_uranus/j9q308lyq_flt.fits acs/10805_uranus/j9q301g3q_flt.fits acs/10805_uranus/j9q301fyq_flt.fits acs/10805_uranus/j9q301g1q_flt.fits acs/10805_uranus/j9q310mkq_flt.fits acs/10805_uranus/j9q310mhq_flt.fits acs/10805_uranus/j9q310mlq_flt.fits acs/10805_uranus/j9q308m0q_flt.fits acs/10805_uranus/j9q308m2q_flt.fits acs/10805_uranus/j9q308lvq_flt.fits acs/10805_uranus/j9q306l2q_flt.fits acs/10805_uranus/j9q306l7q_flt.fits acs/10805_uranus/j9q310mmq_flt.fits acs/10805_uranus/j9q306l6q_flt.fits acs/10805_uranus/j9q306l1q_flt.fits acs/10805_uranus/j9q308m1q_flt.fits acs/10805_uranus/j9q301fzq_flt.fits acs/10805_uranus/j9q306l4q_flt.fits acs/10860_2002ux25/j9qs12hrq_flt.fits acs/10860_2002ux25/j9qs12hsq_flt.fits acs/10800_01og109/j9rpa3ghq_flt.fits acs/10800_01og109/j9rpa3ggq_flt.fits acs/10800_01og109/j9rpa3giq_flt.fits acs/10800_01og109/j9rpa3gjq_flt.fits acs/10514_any/j9fw33qzq_flt.fits acs/10514_any/j9fw33r1q_flt.fits acs/10514_any/j9fw05scq_flt.fits acs/10514_any/j9fw33r4q_flt.fits acs/10514_any/j9fw05s9q_flt.fits acs/10514_any/j9fw05s7q_flt.fits acs/10800_04pa112/j9rpd2n4q_flt.fits acs/10545_ixion/j9fs14wcq_flt.fits acs/10545_ixion/j9fs14wfq_flt.fits acs/10545_ixion/j9fs14weq_flt.fits acs/10545_ixion/j9fs14wdq_flt.fits acs/10508_2002cr46b/j9f608yvq_flt.fits acs/10508_2002cr46b/j9f608yxq_flt.fits acs/10508_2002cr46b/j9f608yzq_flt.fits acs/10508_2002cr46b/j9f608ywq_flt.fits acs/10508_2002cr46b/j9f608z0q_flt.fits acs/10508_2002cr46b/j9f608yyq_flt.fits acs/10508_2002cr46b/j9f608yuq_flt.fits acs/10508_2002cr46b/j9f608z1q_flt.fits acs/10800_05ef304/j9rp26qaq_flt.fits acs/10800_05ef304/j9rp26q9q_flt.fits acs/10800_05ef304/j9rp26q8q_flt.fits acs/10800_05ef304/j9rp26qbq_flt.fits acs/10557_asteroid-7719/j9f304xcq_flt.fit acs/10557_asteroid-7719/j9f304xbq_flt.fit acs/10514_82075/j9fw05s6q_flt.fits acs/10514_82075/j9fw05sbq_flt.fits acs/10514_82075/j9fw05s8q_flt.fits acs/10514_99rb216/j9fw33r0q_flt.fits acs/10514_99rb216/j9fw33qyq_flt.fits acs/10514_99rb216/j9fw33r3q_flt.fits
The unknown problem:
wfc3/12537_any/ibu521krq_flt.fits wfc3/12537_any/ibu521kvq_flt.fits wfc3/12537_any/ibu521kyq_flt.fits
In Ticket #134, we looked at these last three images and determined they were of little value. The value of the other 80 is harder to determine, although having an
EXPTIMEflag of 0 likely indicates they are bad observations.One way to deal with these files in the pipeline is to have something that checks the proposalid (like j9fw33r3q) against this list, and to abort after the cosmic-ray rejection step if it finds it. That's effectively what the pipeline already does, but this way we can throw in an error message into the log that this is a known bad file.
I'll create a 'bad-files' branch to add this change.