search

lukeshingles / atlasserver

ATLAS forced photometry server
https://fallingstar-data.com/forcedphot/
MIT License
9 stars 0 forks source link

possible wrong MJD entry #57

Open peanutr9477 opened 1 week ago

peanutr9477 commented 1 week ago

Hi, thanks for the work of providing this amazing database. I encounter a problem when I try to get a forced photometry light curve of a source at the coordinate of 3.84012, -38.90534 It appears that there is a double entry at 59748.241863MJD with the same camera and same filter. The fits headers of the two images show that they are most likely two exposures at different time, but they have identical MJD-OBS entries. Could you investigate what the cause of this problem is? I hope this won't affect the value of other MJD entries.

BTW, this double entry won't appear if you only query just +-1 of that day. It will only appear with a longer duration criterion.

genghisken commented 1 week ago

Hi! Many thanks for bringing this to our attention. Digging into the data implies that there are problems with duplicate MJDs at ATLAS-CHL (04a) on the night of 59748. Going deeper, I spotted some duplicates at ATLAS-HKO (02a) on 59998 and at ATLAS-STH (03a) on night 59844. I've raised this with our colleagues. I'm not sure how this will be resolved yet, so beware of those two other nights for the time being.

peanutr9477 commented 1 week ago

Thank you for looking into this problem. I found 19 more sources facing the same issue that my team downloaded. I would like to know to what extent this damage is. Are these MJD entries wrong for time only but they are still correct for the date? Or is it possible that this problem could affect their date record as well?

genghisken commented 1 week ago

As far as I can see, ONLY the 3 MJDs and telescopes are affected as stated above. Your 19 other sources almost certainly have duplicates on the same MJDs. If you see something different, let us know, and send us the Job IDs, so we can confirm the problem if necessary. Data is not "damaged". We are reviewing possible fixes, but no promises yet.

peanutr9477 commented 6 days ago

I see. I have a bit closer look into the downloaded data. I find that all sources having duplicated timestamp have the record of 59748.241863MJD and they have coordinates ranged in 357deg < RA < 29deg. Their observed IDs are 04a59748o0619o, 04a59748o0620o, 04a59748o0624o, 04a59748o0629o, 04a59748o0630o, 04a59748o0631o, 04a59748o0660o, 04a59748o0665o, 04a59748o0666o, 04a59748o0667o, 04a59748o0668o, 04a59748o0669o, 04a59748o0676o, 04a59748o0677o, 04a59748o0705o, 04a59748o0706o, 04a59748o0707o, 04a59748o0708o, 04a59748o0711o, 04a59748o0712o.

I also examine the data we downloaded for 03a59843, 03a59844, 03a59845, 02a59998, and 02a59999, but find no duplications for a same source. Could you specify a bit more detail on what kind of data I should avoid for these two days?

smarttgit commented 3 days ago

After a review of this at our weekly ATLAS call, it looks like a lot of the data from these nights has been corrupted by a shutter problem. This results in the shutter open/close time and hence the MJD-OBS going awol. We looked at the data and would recommend avoiding use of these data with duplciated MJD are not used scientifically.

smarttgit commented 3 days ago

The MJD stamps probably won't get fixed anytime soon, since it appears the images are not worth fixing. They appear trailed, very poor zero point, or otherwise corrupted due to the shutter issue. In effect the duplciate MJD is a flag not to use.

peanutr9477 commented 3 days ago

Thank you for providing this information. I am happy to simply discard them. Could you provide a more detail list of corrupted observations? I notice the observation ID is sometimes +-1 day from the MJD record. Therefore, I am not certain which observations are affected due to this problem.