Open howardisaacson opened 9 months ago
awhoward
commented
9 months ago That's interesting about Green or Red being missing in these observations. My recollection is that the issue is related to the L0 assembler, something about a default value of Object = HD10700 not being updated in edge cases. @bpholden, thoughts and musings?
I agree that this is confusing because Tau Ceti is a star that we care a lot about (although we use '10700' as the Object name).
awhoward
commented
9 months ago One other thought is that the observations with Green and Red selected, but one of them missing, are flawed. How much effort do we want to expend fixing them? I can see arguments both ways. Thoughts, @howardisaacson ?
bpholden
commented
9 months ago I looked into this yesterday, and I have looked again.
There are no FITS header keywords with the value 10700 in any L0 files from the past few days.
So where does the value for OBJECT in Jump come from?
The particular file that Howard links to has ( https://jump.caltech.edu/observing-logs/kpf/KP.20240106.30701.07) :
OBJECT = 'autocal-etalon-all-night' / Object TARGNAME= '' / KPF Target name DCSNAME = '2022acko_S1' / DCS Target name - may not make any sense FULLTARG= '' / Full Target name from kpfconfig GAIAID = '' / GAIA Target name from kpfconfig 2MASSID = '' / 2MASS Target name from kpfconfig
On Wed, Jan 10, 2024 at 11:17 AM Andrew Howard @.***> wrote:
That's interesting about Green or Red being missing in these observations. My recollection is that the issue is related to the L0 assembler, something about a default value of Object = HD10700 not being updated in edge cases. @bpholden https://github.com/bpholden, thoughts and musings?
I agree that this is confusing because Tau Ceti is a star that we care a lot about (although we use '10700' as the Object name).
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885529013, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGTEMVSFUP2Z7HN3WFDYN3SOBAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGUZDSMBRGM . You are receiving this because you were mentioned.Message ID: @.*** com>
awhoward
commented
9 months ago This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
# HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
# HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
# HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
awhoward
commented
9 months ago I bet the L1 file starts with a template that has Object = HD10700.
bjfultn
commented
9 months ago This is correct. The defaults are defined in kpfpipe/models/metadata/KPF_headers_L?.csv
bpholden
commented
9 months ago There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGVUUINI5K7QABDBD3TYN3YKXAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGY2DAOJRGY . You are receiving this because you were mentioned.Message ID: @.*** com>
howardisaacson
commented
9 months ago I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAJJHGVUUINI5K7QABDBD3TYN3YKXAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGY2DAOJRGY>
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKFHJYTVQCLRWITZP4NG5TYN32NRAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGY3DGNBRG4 . You are receiving this because you were mentioned.Message ID: @.*** com>
bpholden
commented
8 months ago OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAKFHJYTVQCLRWITZP4NG5TYN32NRAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGY3DGNBRG4>
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGT4SXFKPCLK4XGDOB3YN32YBAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGY3DONRQGE . You are receiving this because you were mentioned.Message ID: @.*** com>
howardisaacson
commented
8 months ago Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAJJHGT4SXFKPCLK4XGDOB3YN32YBAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGY3DONRQGE>
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKFHJ2VHFM7NLXDUG4IAW3YOBAGZAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXHA3TKMJQG4 . You are receiving this because you were mentioned.Message ID: @.*** com>
howardisaacson
commented
8 months ago @bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAJJHGT4SXFKPCLK4XGDOB3YN32YBAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGY3DONRQGE>
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKFHJ2VHFM7NLXDUG4IAW3YOBAGZAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXHA3TKMJQG4 . You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
bpholden
commented
8 months ago That is ... odd.
The RED is yes because the file exists. The file was not incorporated into the final L0 and no reason is given in the log. It is supposed to say why if there was a failure.
On Thu, Jan 11, 2024 at 5:20 PM Howard Isaacson @.***> wrote:
@bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAKFHJ2VHFM7NLXDUG4IAW3YOBAGZAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXHA3TKMJQG4>
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888246278, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGUZNOLLHVE5GDAKILLYOCFXTAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGI2DMMRXHA . You are receiving this because you were mentioned.Message ID: @.*** com>
bpholden
commented
8 months ago I have added more logging and removed a logic error.
The red file was not included because the FRAMENO in the header did not match the FRAMENO of either the exposure or of the filename of the red data. That is odd as well. I did not log that error, however, and that is now logged at least.
On Thu, Jan 11, 2024 at 5:29 PM Brad Holden @.***> wrote:
That is ... odd.
The RED is yes because the file exists. The file was not incorporated into the final L0 and no reason is given in the log. It is supposed to say why if there was a failure.
On Thu, Jan 11, 2024 at 5:20 PM Howard Isaacson @.***> wrote:
@bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAKFHJ2VHFM7NLXDUG4IAW3YOBAGZAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXHA3TKMJQG4>
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888246278, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGUZNOLLHVE5GDAKILLYOCFXTAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGI2DMMRXHA . You are receiving this because you were mentioned.Message ID: @.*** com>
howardisaacson
commented
8 months ago The occurrence of this error state (red CCD missing) seems to be more rare since last week. Just today there was one more: KP.20240116.65837.83 https://jump.caltech.edu/observing-logs/kpf/KP.20240116.65837.83
On Thu, Jan 11, 2024 at 5:54 PM Brad Holden @.***> wrote:
I have added more logging and removed a logic error.
The red file was not included because the FRAMENO in the header did not match the FRAMENO of either the exposure or of the filename of the red data. That is odd as well. I did not log that error, however, and that is now logged at least.
On Thu, Jan 11, 2024 at 5:29 PM Brad Holden @.***> wrote:
That is ... odd.
The RED is yes because the file exists. The file was not incorporated into the final L0 and no reason is given in the log. It is supposed to say why if there was a failure.
On Thu, Jan 11, 2024 at 5:20 PM Howard Isaacson @.***> wrote:
@bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in /data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits:
OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888246278>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAJJHGUZNOLLHVE5GDAKILLYOCFXTAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGI2DMMRXHA>
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888291185, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKFHJYAKJJMSY3SCLDNG4TYOCJV3AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGI4TCMJYGU . You are receiving this because you were mentioned.Message ID: @.*** com>
bpholden
commented
8 months ago When running the l0_assemble on the file showing the error, the header keywords report the correct values:
SIMPLE = T / conforms to FITS standard BITPIX = 8 / array data type NAXIS = 0 / number of array dimensions EXTEND = T FAVER = '0 ' / Written by FITSAssemble version 1.4 TIMESYS = 'UTC ' CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'NO ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found?
which obviously does not match the version produced by the kpfassemble dispatcher.
I will restart the assembler, after reinstalling the software.
On Tue, Jan 16, 2024 at 1:06 PM Howard Isaacson @.***> wrote:
The occurrence of this error state (red CCD missing) seems to be more rare since last week. Just today there was one more: KP.20240116.65837.83 https://jump.caltech.edu/observing-logs/kpf/KP.20240116.65837.83
On Thu, Jan 11, 2024 at 5:54 PM Brad Holden @.***> wrote:
I have added more logging and removed a logic error.
The red file was not included because the FRAMENO in the header did not match the FRAMENO of either the exposure or of the filename of the red data. That is odd as well. I did not log that error, however, and that is now logged at least.
On Thu, Jan 11, 2024 at 5:29 PM Brad Holden @.***> wrote:
That is ... odd.
The RED is yes because the file exists. The file was not incorporated into the final L0 and no reason is given in the log. It is supposed to say why if there was a failure.
On Thu, Jan 11, 2024 at 5:20 PM Howard Isaacson @.***> wrote:
@bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in /data/kpf/L0/20240106/KP.20240106.30701.07.fits:
OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits: OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888246278>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888291185>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAKFHJYAKJJMSY3SCLDNG4TYOCJV3AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGI4TCMJYGU>
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894512690, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGQFWVJOXBL65OEGYB3YO3TWDAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGUYTENRZGA . You are receiving this because you were mentioned.Message ID: @.*** com>
howardisaacson
commented
8 months ago Two more examples from today that is missing the Red CCD.
KP.20240117.04184.64
CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:09:47.801' / Halfway point of the exposure, unweighted
KP.20240117.04239.41
CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:09:47.801' / Halfway point of the exposure, unweighted
On Tue, Jan 16, 2024 at 1:46 PM Brad Holden @.***> wrote:
When running the l0_assemble on the file showing the error, the header keywords report the correct values:
HDU 0 in KP.20240116.65832.45.fits:
SIMPLE = T / conforms to FITS standard BITPIX = 8 / array data type NAXIS = 0 / number of array dimensions EXTEND = T FAVER = '0 ' / Written by FITSAssemble version 1.4 TIMESYS = 'UTC ' CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'NO ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found?
which obviously does not match the version produced by the kpfassemble dispatcher.
I will restart the assembler, after reinstalling the software.
On Tue, Jan 16, 2024 at 1:06 PM Howard Isaacson @.***> wrote:
The occurrence of this error state (red CCD missing) seems to be more rare since last week. Just today there was one more: KP.20240116.65837.83 https://jump.caltech.edu/observing-logs/kpf/KP.20240116.65837.83
On Thu, Jan 11, 2024 at 5:54 PM Brad Holden @.***> wrote:
I have added more logging and removed a logic error.
The red file was not included because the FRAMENO in the header did not match the FRAMENO of either the exposure or of the filename of the red data. That is odd as well. I did not log that error, however, and that is now logged at least.
On Thu, Jan 11, 2024 at 5:29 PM Brad Holden @.***> wrote:
That is ... odd.
The RED is yes because the file exists. The file was not incorporated into the final L0 and no reason is given in the log. It is supposed to say why if there was a failure.
On Thu, Jan 11, 2024 at 5:20 PM Howard Isaacson @.***> wrote:
@bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in
/data/kpf/L0/20240106/KP.20240106.30701.07.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits: OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888246278>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888291185>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894512690>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAJJHGQFWVJOXBL65OEGYB3YO3TWDAVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGUYTENRZGA>
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894566647, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKFHJZTFBDQBCDYKFU74GTYO3YL7AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGU3DMNRUG4 . You are receiving this because you were mentioned.Message ID: @.*** com>
bpholden
commented
8 months ago 1) Those both happened before the final installation and restart yesterday.
2) When I look at Jump, the second file shows RED='No' https://jump.caltech.edu/observing-logs/kpf/KP.20240117.04239.41
This just adds to the confusion of course.
EXTEND = T FAVER = '0 ' / Written by FITSAssemble version 1.4 TIMESYS = 'UTC ' CAMERAS = 2 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'NO ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:20:39.419' / Halfway point of the
On Wed, Jan 17, 2024 at 4:40 PM Howard Isaacson @.***> wrote:
Two more examples from today that is missing the Red CCD.
KP.20240117.04184.64
CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:09:47.801' / Halfway point of the exposure, unweighted
KP.20240117.04239.41
CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:09:47.801' / Halfway point of the exposure, unweighted
On Tue, Jan 16, 2024 at 1:46 PM Brad Holden @.***> wrote:
When running the l0_assemble on the file showing the error, the header keywords report the correct values:
HDU 0 in KP.20240116.65832.45.fits:
SIMPLE = T / conforms to FITS standard BITPIX = 8 / array data type NAXIS = 0 / number of array dimensions EXTEND = T FAVER = '0 ' / Written by FITSAssemble version 1.4 TIMESYS = 'UTC ' CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'NO ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found?
which obviously does not match the version produced by the kpfassemble dispatcher.
I will restart the assembler, after reinstalling the software.
On Tue, Jan 16, 2024 at 1:06 PM Howard Isaacson @.***> wrote:
The occurrence of this error state (red CCD missing) seems to be more rare since last week. Just today there was one more: KP.20240116.65837.83 https://jump.caltech.edu/observing-logs/kpf/KP.20240116.65837.83
On Thu, Jan 11, 2024 at 5:54 PM Brad Holden @.***> wrote:
I have added more logging and removed a logic error.
The red file was not included because the FRAMENO in the header did not match the FRAMENO of either the exposure or of the filename of the red data. That is odd as well. I did not log that error, however, and that is now logged at least.
On Thu, Jan 11, 2024 at 5:29 PM Brad Holden @.***> wrote:
That is ... odd.
The RED is yes because the file exists. The file was not incorporated into the final L0 and no reason is given in the log. It is supposed to say why if there was a failure.
On Thu, Jan 11, 2024 at 5:20 PM Howard Isaacson @.***> wrote:
@bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in
/data/kpf/L0/20240106/KP.20240106.30701.07.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits: OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888246278>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888291185>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894512690>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894566647>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAKFHJZTFBDQBCDYKFU74GTYO3YL7AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGU3DMNRUG4>
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1897561561, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGSOBKPDPD2K7ITZDALYPBVQ5AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJXGU3DCNJWGE . You are receiving this because you were mentioned.Message ID: @.*** com>
bpholden
commented
8 months ago Found an error today where the Ca HK does not appear but the header lies.
This is post the restart.
On Wed, Jan 17, 2024 at 4:52 PM Brad Holden @.***> wrote:
1) Those both happened before the final installation and restart yesterday.
2) When I look at Jump, the second file shows RED='No' https://jump.caltech.edu/observing-logs/kpf/KP.20240117.04239.41
This just adds to the confusion of course.
EXTEND = T FAVER = '0 ' / Written by FITSAssemble version 1.4 TIMESYS = 'UTC ' CAMERAS = 2 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'NO ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:20:39.419' / Halfway point of the
On Wed, Jan 17, 2024 at 4:40 PM Howard Isaacson @.***> wrote:
Two more examples from today that is missing the Red CCD.
KP.20240117.04184.64
CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:09:47.801' / Halfway point of the exposure, unweighted
KP.20240117.04239.41
CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:09:47.801' / Halfway point of the exposure, unweighted
On Tue, Jan 16, 2024 at 1:46 PM Brad Holden @.***> wrote:
When running the l0_assemble on the file showing the error, the header keywords report the correct values:
HDU 0 in KP.20240116.65832.45.fits:
SIMPLE = T / conforms to FITS standard BITPIX = 8 / array data type NAXIS = 0 / number of array dimensions EXTEND = T FAVER = '0 ' / Written by FITSAssemble version 1.4 TIMESYS = 'UTC ' CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'NO ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found?
which obviously does not match the version produced by the kpfassemble dispatcher.
I will restart the assembler, after reinstalling the software.
On Tue, Jan 16, 2024 at 1:06 PM Howard Isaacson @.***> wrote:
The occurrence of this error state (red CCD missing) seems to be more rare since last week. Just today there was one more: KP.20240116.65837.83 https://jump.caltech.edu/observing-logs/kpf/KP.20240116.65837.83
On Thu, Jan 11, 2024 at 5:54 PM Brad Holden @.***> wrote:
I have added more logging and removed a logic error.
The red file was not included because the FRAMENO in the header did not match the FRAMENO of either the exposure or of the filename of the red data. That is odd as well. I did not log that error, however, and that is now logged at least.
On Thu, Jan 11, 2024 at 5:29 PM Brad Holden @.***> wrote:
That is ... odd.
The RED is yes because the file exists. The file was not incorporated into the final L0 and no reason is given in the log. It is supposed to say why if there was a failure.
On Thu, Jan 11, 2024 at 5:20 PM Howard Isaacson @.***> wrote:
@bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in
/data/kpf/L0/20240106/KP.20240106.30701.07.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits: OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888246278>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888291185>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894512690>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894566647>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAKFHJZTFBDQBCDYKFU74GTYO3YL7AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGU3DMNRUG4>
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1897561561, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGSOBKPDPD2K7ITZDALYPBVQ5AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJXGU3DCNJWGE . You are receiving this because you were mentioned.Message ID: @.*** com>
bpholden
commented
8 months ago OK, the Red CCD failed and two images were impacted last night. The alert came at 12:37 AM PST.
The files are: https://jump.caltech.edu/observing-logs/kpf/KP.20240118.31027.95 https://jump.caltech.edu/observing-logs/kpf/KP.20240118.31082.94
Both correctly state that the RED image is missing. The target is still 10700 and the exposure time is blank.
On Wed, Jan 17, 2024 at 4:56 PM Brad Holden @.***> wrote:
Found an error today where the Ca HK does not appear but the header lies.
This is post the restart.
On Wed, Jan 17, 2024 at 4:52 PM Brad Holden @.***> wrote:
1) Those both happened before the final installation and restart yesterday.
2) When I look at Jump, the second file shows RED='No' https://jump.caltech.edu/observing-logs/kpf/KP.20240117.04239.41
This just adds to the confusion of course.
EXTEND = T FAVER = '0 ' / Written by FITSAssemble version 1.4 TIMESYS = 'UTC ' CAMERAS = 2 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'NO ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:20:39.419' / Halfway point of the
On Wed, Jan 17, 2024 at 4:40 PM Howard Isaacson @.***> wrote:
Two more examples from today that is missing the Red CCD.
KP.20240117.04184.64
CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:09:47.801' / Halfway point of the exposure, unweighted
KP.20240117.04239.41
CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'NO ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-17T01:09:47.801' / Halfway point of the exposure, unweighted
On Tue, Jan 16, 2024 at 1:46 PM Brad Holden @.***> wrote:
When running the l0_assemble on the file showing the error, the header keywords report the correct values:
HDU 0 in KP.20240116.65832.45.fits:
SIMPLE = T / conforms to FITS standard BITPIX = 8 / array data type NAXIS = 0 / number of array dimensions EXTEND = T FAVER = '0 ' / Written by FITSAssemble version 1.4 TIMESYS = 'UTC ' CAMERAS = 3 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'NO ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found?
which obviously does not match the version produced by the kpfassemble dispatcher.
I will restart the assembler, after reinstalling the software.
On Tue, Jan 16, 2024 at 1:06 PM Howard Isaacson @.***> wrote:
The occurrence of this error state (red CCD missing) seems to be more rare since last week. Just today there was one more: KP.20240116.65837.83 https://jump.caltech.edu/observing-logs/kpf/KP.20240116.65837.83
On Thu, Jan 11, 2024 at 5:54 PM Brad Holden @.***> wrote:
I have added more logging and removed a logic error.
The red file was not included because the FRAMENO in the header did not match the FRAMENO of either the exposure or of the filename of the red data. That is odd as well. I did not log that error, however, and that is now logged at least.
On Thu, Jan 11, 2024 at 5:29 PM Brad Holden @.***> wrote:
That is ... odd.
The RED is yes because the file exists. The file was not incorporated into the final L0 and no reason is given in the log. It is supposed to say why if there was a failure.
On Thu, Jan 11, 2024 at 5:20 PM Howard Isaacson @.***> wrote:
@bpholden I see an exposure from today (timestamp is 3:56 PT, on cadence) that is missing the red CCD: KP.20240111.86142.34.fits (and also KP.20240111.86222.12.fits).
Here are some of the header keywords. Am I correct that the "RED" keyword should equal NO for this since it is missing red data?
TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers CAMERAS = 4 / Detectors in assembly request GREEN = 'YES ' / Was this camera found? RED = 'YES ' / Was this camera found? CA_HK = 'YES ' / Was this camera found? EXPMETER= 'YES ' / Was this camera found? GUIDE = 'NO ' / Was this camera found? DATE-MID= '2024-01-11T23:55:42.874' / Halfway point of the exposure, unweighted
On Thu, Jan 11, 2024 at 2:35 PM Howard Isaacson @.***> wrote:
Great! I see "TRIGTARG= 'Green,Red,Ca_HK,ExpMeter' / Cameras that were sent triggers And an individual yes/no keyword for each. I will report back if I find missing red/green ccds and work on a QC test for this.
On Thu, Jan 11, 2024 at 12:00 PM Brad Holden @.***> wrote:
OK, I updated the assembler to report correctly whether or not the files are present. It used to just report whether or not a filename was available. The cases with missing files were ones where the file was not created. The existence of a filename is not a useful one for the user of the data, so I have changed the assembler to report if the file is found in the header.
Going forward, the RED, GREEN, etc. keywords should report YES if those data are attached as HDUs and NO if they are not attached. The TRIG_TARG keyword will still report the intent. Hopefully it will be straightforward to compare the TRIG_TARG with the detector keywords and so flag those observations with missing data.
On Wed, Jan 10, 2024 at 12:28 PM Howard Isaacson @.***> wrote:
I have been marking files with either red or green data that are missing as Junk. Maybe we can add this as a QC routine to do it automatically.
-Howard
On Wed, Jan 10, 2024 at 12:26 PM Brad Holden @.***> wrote:
There is still the issue of missing data. The L0 header claims to have found all of the files, yet one of the files is not incorporated.
That is a bug on the part of the assembler.
On Wed, Jan 10, 2024 at 12:08 PM Andrew Howard @.***> wrote:
This is weird.
shrek% fitsheader -k Object -e 0 /data/kpf/??/20240106/KP.20240106.30701.07*.fits
HDU 0 in
/data/kpf/L0/20240106/KP.20240106.30701.07.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/2D/20240106/KP.20240106.30701.07_2D.fits: OBJECT = 'autocal-etalon-all-night' / Object
HDU 0 in
/data/kpf/L1/20240106/KP.20240106.30701.07_L1.fits: OBJECT = 'HD10700 ' / Object Name
So the Object name is changed in the 2D -> L1 processing and then read into Jump.
Sorry to cast aspersions on the L0 Assembler!
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885640916>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885663417>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1885667601>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1887875107>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888246278>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1888291185>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub <
https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894512690>,
or unsubscribe <
. You are receiving this because you were mentioned.Message ID: @.*** com>
— Reply to this email directly, view it on GitHub < https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1894566647>,
or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAKFHJZTFBDQBCDYKFU74GTYO3YL7AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGU3DMNRUG4>
. You are receiving this because you were mentioned.Message ID: @.*** com>
--
Howard Isaacson Research Scientist in Astronomy University of California, Berkeley
— Reply to this email directly, view it on GitHub https://github.com/Keck-DataReductionPipelines/KPF-Pipeline/issues/773#issuecomment-1897561561, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJJHGSOBKPDPD2K7ITZDALYPBVQ5AVCNFSM6AAAAABBT5IYJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJXGU3DCNJWGE . You are receiving this because you were mentioned.Message ID: @.*** com>
An object name of HD10700 seems to be a fill-in when something goes wrong. Here is one example: https://jump.caltech.edu/observing-logs/kpf/KP.20240106.30701.07. Spot checking these shows that either the green or red CCD is missing.