Difference image query linking to many broken files

[nrgolovich]

I'm querying for difference images with a simple search for files near an ra/dec point between two times. The majority of the files in my zquery.metatable do not download, and the linked files do not exist on the server.

zquery.load_metadata(radec=[182.43061649701434,-1.0493840126477587], size=15, sql_query="obsjd>2458447.25 and obsjd<2458448.25")

Only 3/53 files download with the above query.

[simeonreusch]

Have you ruled out an authorization issue?

[nrgolovich]

All of the 53 have ipac_gid==1. I have gathered that means they are public data. Is this what you meant?

[simeonreusch]

Unfortunately, public can also be mean "is in principle public, but only published in 1-year intervals". Therefore the 2019 MSIP data is not accessible yet.

[fmasci]

MSIP data taken up to Jun 30, 2019 was made public about 1 month ago (DR2). The next release (DR3 in June 2020) will include data taken up to Dec 31, 2019.

Regards, Frank

---

From: simeonreusch notifications@github.com Sent: Saturday, January 18, 2020 4:32:18 AM To: MickaelRigault/ztfquery ztfquery@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [MickaelRigault/ztfquery] Difference image query linking to many broken files (#22)

Unfortunately, public can also be mean "is in principle public, but only published in 1-year intervals". Therefore the 2019 MSIP data is not accessible yet.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/MickaelRigault/ztfquery/issues/22?email_source=notifications&email_token=ADXUBINXC3JZTAHCUZUOLBLQ6LZFFA5CNFSM4KIQF5BKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJJXIRQ#issuecomment-575894598, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADXUBIJ2V3WXASLUZVGVJ63Q6LZFFANCNFSM4KIQF5BA.

[nrgolovich]

The data I am after was taken in Nov 2018 - Feb 2019, so that shouldn't be it then.

[MickaelRigault]

Hi Nate,

Indeed, many image difference files you want do not exist.

@fmasci: For the sci/2018/1121/478484/ , why does the scimrefdiffimg exist for zr_c15_o_q4 and not for zr_c11_o_q4 ? (so CCD 15 ok, but not CCD 11)

[fmasci]

Both those quadrants do not have subtractions because their reference images were created on 2018-11-27 (6 days later). Here are the subtractions that exist for this exposure, simply because a production quality reference existed:

ztf_20181121478484_000471_zr_c01_o_q1_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c01_o_q4_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c12_o_q2_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c13_o_q1_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c13_o_q2_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c13_o_q4_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c14_o_q1_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c14_o_q2_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c15_o_q1_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c15_o_q2_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c15_o_q3_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c16_o_q1_scimrefdiffimg.fits.fz ztf_20181121478484_000471_zr_c16_o_q2_scimrefdiffimg.fits.fz

Note: we plan to backfill all subtractions for which no reference existed during realtime processing. Stay tuned.

Regards, Frank

On Jan 20, 2020, at 1:23 AM, Mickael Rigault notifications@github.com<mailto:notifications@github.com> wrote:

Hi Nate,

Indeed, many imagine difference files you want do not exist.

@fmascihttps://github.com/fmasci: For the sci/2018/1121/478484/ , why does the scimrefdiffimg exist for zr_c15_o_q4 and not for zr_c11_o_q4 ? (so CCD 15 ok, but not CCD 11)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/MickaelRigault/ztfquery/issues/22?email_source=notifications&email_token=ADXUBIMVOQP3XHT2RZVEUSDQ6VUSXA5CNFSM4KIQF5BKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJL5XKY#issuecomment-576183211, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADXUBIOLDWPSJAQUVYUF3PLQ6VUSXANCNFSM4KIQF5BA.

[nrgolovich]

Great, thanks for the information. I guess I'll proceed with my analysis of the data I can get now and check back later.