EranOfek / AstroPack

Astronomy & Astrophysics Software Pacakge
Other
17 stars 4 forks source link

pipeline.DemonLAST / fwhm #199

Closed YarinMS closed 1 year ago

YarinMS commented 1 year ago

In order to use the forced photometry tool, we would like to run the pipeline with an option to save the proc images as well, by executing :

image

It looks like the pipeline fails for all the images, although they look ok (to me). For this report I've used images from : group 1 - First image: /last04w/data2/archive/LAST.01.04.04/new/LAST.01.04.04_20230720.230849.762_clear_264+65_020_001_001_sci_raw_Image_1.fits up to group 3 - First image: /last04w/data2/archive/LAST.01.04.04/new/LAST.01.04.04_20230720.215515.584_clear_264+65_020_001_001_sci_raw_Image_1.fits

For the time being I'll leave these images in the failed directory, them and only them. In case anyone would like to examine them. A failed image to demonstrate,

image

While running the pipeline it fails in processing all the groups. We get the following try error: [ERR] pipeline.DemonLAST try error: Subscripted assignment between dissimilar structures. / funname: AstroPSF.curve_of_growth @ line: 432

image

Eran asked to check what is the content of the PSFData in the Coadd AstroImage BEFORE the call to fhwm, to do so, we run the pipeline on debug mode and taking the next break points:

image

Which uses the function,

pipeline.generic.multiRaw2procCoadd

Next, we are sent to,

pipeline.generic.singleRaw2proc

image

Then, we encounter the first call to fwhm :

image

Assuming the content we are looking for lies in : SI.PPSFData We get a non-empty AstroPSF object :

image

Looking further, SI.PSFData.Data or SI.PSFData.DataPSF demands to choose a subframe (out of 24), looking at an arbitrary sub image (15) around the center of the detector :

SI.PSFData.Data == SI.PSFData.DataPSF and it looks like the PSF spreads the light in a gaussian manner, (as expected?) :

image

And the lower part :

image

Making sure it is normalized.

So, assuming I understood what Eran asked, the problem originates deeper in AstroPSF and specifically in imUtil.psf.curve_of_growth

EranOfek commented 1 year ago

Hi Yarin,

Please send me the PSF matrix in mat or ascii file so I can test it. Note that sending mat files via the WIS network is not possible (send it to my gmail). Thanks, Eran

On Tue, Jul 25, 2023 at 9:10 PM YarinMS @.***> wrote:

In order to use the forced photometry tool, we would like to run the pipeline with an option to save the proc images as well, by executing : [image: image] https://user-images.githubusercontent.com/118757319/255983141-74aebdc0-03b7-4256-ba1f-914c4514b50e.png

It looks like the pipeline fails for all the images, although they look ok (to me). For this report I've used images from : group 1 - First image: /last04w/data2/archive/LAST.01.04.04/new/LAST.01.04.04_20230720.230849.762_clear_264+65_020_001_001_sci_raw_Image_1.fits

up to group 3 - First image: /last04w/data2/archive/LAST.01.04.04/new/LAST.01.04.04_20230720.215515.584_clear_264+65_020_001_001_sci_raw_Image_1.fits

For the time being I'll leave these images in the failed directory, them and only them. In case anyone would like to examine them. A failed image to demonstrate, [image: image] https://user-images.githubusercontent.com/118757319/255985524-f80befb6-fe8e-4f2b-90b8-00d3f23b8906.png

While running the pipeline it fails in processing all the groups. We get the following try error: [ERR] pipeline.DemonLAST try error: Subscripted assignment between dissimilar structures. / funname: AstroPSF.curve_of_growth @ line: 432 [image: image] https://user-images.githubusercontent.com/118757319/255982710-ae18e6b0-8b2d-41bc-a29f-745c9590610d.png

Eran asked to check what is the content of the PSFData in the Coadd AstroImage BEFORE the call to fhwm, to do so, we run the pipeline on debug mode and taking the next break points: [image: image] https://user-images.githubusercontent.com/118757319/255986609-8cb6a757-ed6a-413b-8455-57e34d54828c.png Which uses the function,

pipeline.generic.multiRaw2procCoadd

Next, we are sent to,

pipeline.generic.singleRaw2proc [image: image] https://user-images.githubusercontent.com/118757319/255987252-fe2669bb-e34d-42d0-a56f-b71e68d8d0d9.png Then, we encounter the first call to fwhm : [image: image] https://user-images.githubusercontent.com/118757319/255987561-dd7464ef-b103-479f-a0f2-7224188dad58.png

Assuming the content we are looking for lies in : SI.PPSFData We get a non-empty AstroPSF object : [image: image] https://user-images.githubusercontent.com/118757319/255988108-6dfb0a42-47c3-44f4-b53c-10cbbf72597b.png

Looking further, SI.PSFData.Data or SI.PSFData.DataPSF demands to choose a subframe (out of 24), looking at an arbitrary sub image (15) around the center of the detector :

SI.PSFData.Data == SI.PSFData.DataPSF and it looks like the PSF spreads the light in a gaussian manner, (as expected?) : [image: image] https://user-images.githubusercontent.com/118757319/255989112-e7069f4f-526f-4b2d-9b7e-bbf492987613.png And the lower part : [image: image] https://user-images.githubusercontent.com/118757319/255989237-5e280bd3-4681-4bf9-91e2-cf5f568d23c0.png Making sure it is normalized.

So, assuming I understood what Eran asked, the problem originates deeper in AstroPSF and specifically in imUtil.psf.curve_of_growth

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/199, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4LR2HCVACDU4N6CPBDXSADYVANCNFSM6AAAAAA2XOEDFY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

YarinMS commented 1 year ago

Sure, I’m currently away, I’ll do at as I get back to my computer.

Yarin


From: EranOfek @.> Sent: Tuesday, July 25, 2023 10:36:09 PM To: EranOfek/AstroPack @.> Cc: Yarin-Meir Shani @.>; Author @.> Subject: Re: [EranOfek/AstroPack] pipeline.DemonLAST try error. (Issue #199)

Hi Yarin,

Please send me the PSF matrix in mat or ascii file so I can test it. Note that sending mat files via the WIS network is not possible (send it to my gmail). Thanks, Eran

On Tue, Jul 25, 2023 at 9:10 PM YarinMS @.***> wrote:

In order to use the forced photometry tool, we would like to run the pipeline with an option to save the proc images as well, by executing : [image: image] https://user-images.githubusercontent.com/118757319/255983141-74aebdc0-03b7-4256-ba1f-914c4514b50e.png

It looks like the pipeline fails for all the images, although they look ok (to me). For this report I've used images from : group 1 - First image: /last04w/data2/archive/LAST.01.04.04/new/LAST.01.04.04_20230720.230849.762_clear_264+65_020_001_001_sci_raw_Image_1.fits

up to group 3 - First image: /last04w/data2/archive/LAST.01.04.04/new/LAST.01.04.04_20230720.215515.584_clear_264+65_020_001_001_sci_raw_Image_1.fits

For the time being I'll leave these images in the failed directory, them and only them. In case anyone would like to examine them. A failed image to demonstrate, [image: image] https://user-images.githubusercontent.com/118757319/255985524-f80befb6-fe8e-4f2b-90b8-00d3f23b8906.png

While running the pipeline it fails in processing all the groups. We get the following try error: [ERR] pipeline.DemonLAST try error: Subscripted assignment between dissimilar structures. / funname: AstroPSF.curve_of_growth @ line: 432 [image: image] https://user-images.githubusercontent.com/118757319/255982710-ae18e6b0-8b2d-41bc-a29f-745c9590610d.png

Eran asked to check what is the content of the PSFData in the Coadd AstroImage BEFORE the call to fhwm, to do so, we run the pipeline on debug mode and taking the next break points: [image: image] https://user-images.githubusercontent.com/118757319/255986609-8cb6a757-ed6a-413b-8455-57e34d54828c.png Which uses the function,

pipeline.generic.multiRaw2procCoadd

Next, we are sent to,

pipeline.generic.singleRaw2proc [image: image] https://user-images.githubusercontent.com/118757319/255987252-fe2669bb-e34d-42d0-a56f-b71e68d8d0d9.png Then, we encounter the first call to fwhm : [image: image] https://user-images.githubusercontent.com/118757319/255987561-dd7464ef-b103-479f-a0f2-7224188dad58.png

Assuming the content we are looking for lies in : SI.PPSFData We get a non-empty AstroPSF object : [image: image] https://user-images.githubusercontent.com/118757319/255988108-6dfb0a42-47c3-44f4-b53c-10cbbf72597b.png

Looking further, SI.PSFData.Data or SI.PSFData.DataPSF demands to choose a subframe (out of 24), looking at an arbitrary sub image (15) around the center of the detector :

SI.PSFData.Data == SI.PSFData.DataPSF and it looks like the PSF spreads the light in a gaussian manner, (as expected?) : [image: image] https://user-images.githubusercontent.com/118757319/255989112-e7069f4f-526f-4b2d-9b7e-bbf492987613.png And the lower part : [image: image] https://user-images.githubusercontent.com/118757319/255989237-5e280bd3-4681-4bf9-91e2-cf5f568d23c0.png Making sure it is normalized.

So, assuming I understood what Eran asked, the problem originates deeper in AstroPSF and specifically in imUtil.psf.curve_of_growth

— Reply to this email directly, view it on GitHub https://github.com/EranOfek/AstroPack/issues/199, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUQ4LR2HCVACDU4N6CPBDXSADYVANCNFSM6AAAAAA2XOEDFY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHubhttps://github.com/EranOfek/AstroPack/issues/199#issuecomment-1650421479, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A4KBPR2GGGCOJUFRILVFRETXSAN2TANCNFSM6AAAAAA2XOEDFY. You are receiving this because you authored the thread.Message ID: @.***>

EranOfek commented 1 year ago

The bug was due to another bug fix I introduced a few days ago.
Line 425 in AstroPSF now reads: Result = struct('Radius',cell(Nobj,1), 'Sum',cell(Nobj,1), 'Npix',cell(Nobj,1), 'Mean',cell(Nobj,1), 'Med',cell(Nobj,1), 'CumSum',cell(Nobj,1));