Problems with FindOverlap.py

[SeanPoints]

I was able to use FindOverlap.py without trouble on fields SMC_c01, SMC_c02, SMC_c03, SMC_c04, SMC_c05, LMC_c01, LMC_c02, and LMC_c03. When I try to run on fields LMC_c30 or LMC_c32 I get the following errors: (kred) spoints-ld1[36:workspace10] FindOverlaps.py -all LMC_c32

Starting Field LMC_c32 Tile T01 Traceback (most recent call last): File "/home/points/kred/py_progs/FindOverlaps.py", line 281, in steer(sys.argv) File "/home/points/kred/py_progs/FindOverlaps.py", line 271, in steer do_one_tile(field,one) File "/home/points/kred/py_progs/FindOverlaps.py", line 217, in do_one_tile x=do_one_filter(field,tile,xfilter=one) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/points/kred/py_progs/FindOverlaps.py", line 196, in do_one_filter xx=vstack([xx,xout]) ^^ UnboundLocalError: cannot access local variable 'xx' where it is not associated with a value

Current commit at: 1e1edd6e9baf408d6344730dd3447647b6a55788

[kslong]

Is this a field where you have removed S7 or any images for too large a psf?

From: Sean Points @.> Reply-To: kslong/kred @.> Date: Saturday, February 17, 2024 at 3:00 PM To: kslong/kred @.> Cc: Subscribed @.> Subject: [kslong/kred] Problems with FindOverlap.py (Issue #20)

I was able to use FindOverlap.py without trouble on fields SMC_c01, SMC_c02, SMC_c03, SMC_c04, SMC_c05, LMC_c01, LMC_c02, and LMC_c03. When I try to run on fields LMC_c30 or LMC_c32 I get the following errors: (kred) spoints-ld1[36:workspace10] FindOverlaps.py -all LMC_c32

Starting Field LMC_c32 Tile T01 Traceback (most recent call last): File "/home/points/kred/py_progs/FindOverlaps.py", line 281, in steer(sys.argv) File "/home/points/kred/py_progs/FindOverlaps.py", line 271, in steer do_one_tile(field,one) File "/home/points/kred/py_progs/FindOverlaps.py", line 217, in do_one_tile x=do_one_filter(field,tile,xfilter=one) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/points/kred/py_progs/FindOverlaps.py", line 196, in do_one_filter xx=vstack([xx,xout]) ^^ UnboundLocalError: cannot access local variable 'xx' where it is not associated with a value

Current commit at: 1e1edd6https://urldefense.com/v3/__https:/github.com/kslong/kred/commit/1e1edd6e9baf408d6344730dd3447647b6a55788__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!3wB2PwA3eJ5PRqlJfn4Z6sJz6D5C7OGSRZzzmX0U-Ob1Dp6iNNVc6SXXdDHsF27a6MJr99n_OBm1QdMOF16wxg$

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/kslong/kred/issues/20__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!3wB2PwA3eJ5PRqlJfn4Z6sJz6D5C7OGSRZzzmX0U-Ob1Dp6iNNVc6SXXdDHsF27a6MJr99n_OBm1QdONDhG4Aw$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AATJ4VMX5K3PFITHALRLE5TYUEK6PAVCNFSM6AAAAABDNSOSACVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE2DANRRGUZDAMI__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!3wB2PwA3eJ5PRqlJfn4Z6sJz6D5C7OGSRZzzmX0U-Ob1Dp6iNNVc6SXXdDHsF27a6MJr99n_OBm1QdMIj6kAQA$. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[SeanPoints]

For all, SMC_c01, SMC_c02, SMC_c03, SMC_c04, SMC_c05, LMC_c01, LMC_c02, LMC_c03 I had removed all with seeing > 1.4" and S7. When trying on LMC_c30 and LMC_c32, I had just removed the bad seeing data by removing S7 from the Summary/LMC_c??/LMC_c??.det.tab file.

[kslong]

Please post here the det.tab files you used when the error occurred.

[kslong]

@SeanPoints I was unable to reproduce the error on my own. I need the det.tab file to do more.

[kslong]

... for LMC_c30 at least

[SeanPoints]

LMC_c30_det.tab.txt

File uploaded

[kslong]

@Sean I am unable to reproduce the problem for LMC_c30 using the provided det file. FindOverlaps.py uses tables that are in the Summary directory; it does not access fits files.

I've attached a tar file that contains the version of the Summary directory that I created for testing FindOverlaps. There are instructions in the Readme file on how to run Find Overlaps there. Please see if you can reproduce the error that way.

Then please tar and upload the Summary directory where you found the problem, so I can test it on my linux box.

z.bug20_overlap.240218.tar.gz

Hopefully, that will let me diagnose what is going wrong.

[SeanPoints]

This is odd. I can process LMC_c01, LMC_c02, LMC_c03 without trouble. I'll send files later as Mondays are meeting heavy for me. What is your current commit? Mine is 18c30fbb8b0880ac3af9f2c596888d3b2e5106a6

[SeanPoints]

Error not reproduced running the DoTest LMC_c30 command from your tar file in the bug overlap directory.

[kslong]

Earlier you said you were using https://github.com/kslong/kred/commit/1e1edd6e9baf408d6344730dd3447647b6a55788 which is the one you should be using. The commit https://github.com/kslong/kred/commit/18c30fbb8b0880ac3af9f2c596888d3b2e5106a6 on the main branch is old, and not the one you should be using. Please clarify.

[SeanPoints]

The log file for the successful SMC fields is commit 1e1edd6e9baf408d6344730dd3447647b6a55788 The log file for the fields with some failures is 18c30fbb8b0880ac3af9f2c596888d3b2e5106a6

I don't know how this happened. What is current branch that I need?

[SeanPoints]

I have the dev branch checked out that should be version 1e1edd6, but when I run MefSum.py, it is listing my current commit as 18c30fb. How can I easily change this since I have done something obviously bad.

[kslong]

Did you pull the branch

git checkout dev

git pull origin dev

[SeanPoints]

(kred) spoints-ld1[26:kred] git pull origin dev From https://github.com/kslong/kred

• branch dev -> FETCH_HEAD Auto-merging py_progs/SwarpSetup.py CONFLICT (content): Merge conflict in py_progs/SwarpSetup.py Auto-merging py_progs/Swarp.py Automatic merge failed; fix conflicts and then commit the result.

[SeanPoints]

Back in 1e dev branch. Will try all fields again to see if I get error.

[SeanPoints]

FindOverlaps.py fails if there is only one exposure for a given filter and exposure time combination. It executes fine if there are 0 exposures or 2+ exposures for the given FILTER,EXPTIME combination

[kslong]

The program no longer fails if this is the case, but there is an issue of what to do exactly when there is a file that does not overlap with anything in the downstream programs.

[kslong]

@SeanPoints I am closing this issue as the routine no longer "falls over" if it encounters a situation with a single exposure. A warning is put in the log file concerning this, but this single exposure will not be included in any downstream processing. If you wish an alternative solution, you will need to specify it.