search

LM-SAL / aiapy

Python library for AIA data analysis
https://aiapy.readthedocs.io/en/stable/
BSD 3-Clause "New" or "Revised" License
7 stars 3 forks source link

get_pointing_table failing #71

Closed nabobalis closed 11 months ago

nabobalis commented 3 years ago

In GitLab by @dstansby1 on Nov 4, 2020, 10:24

Trying to update the pointing of a map, I am currently seeing errors that look like

  File "/Users/dstansby/github/solarsynoptic/aia_helpers.py", line 91, in prep
    m = update_pointing(m)
  File "/Users/dstansby/miniconda3/lib/python3.8/site-packages/aiapy/calibrate/meta.py", line 88, in update_pointing
    pointing_table = get_pointing_table(smap.date - 3*u.h, smap.date + 3*u.h)
  File "/Users/dstansby/miniconda3/lib/python3.8/site-packages/aiapy/calibrate/util.py", line 141, in get_pointing_table
    q = jsoc.JSOCClient().search_metadata(
  File "/Users/dstansby/miniconda3/lib/python3.8/site-packages/sunpy/net/jsoc/jsoc.py", line 409, in search_metadata
    res = res.append(self._lookup_records(iargs))
  File "/Users/dstansby/miniconda3/lib/python3.8/site-packages/sunpy/net/jsoc/jsoc.py", line 878, in _lookup_records
    r = c.query(ds, key=key, rec_index=isMeta)
  File "/Users/dstansby/miniconda3/lib/python3.8/site-packages/drms/client.py", line 1119, in query
    index = [it['name'] for it in lres['recinfo']]
KeyError: 'recinfo'

These can be reproduced with 

from aiapy.calibrate.util import get_pointing_table
from astropy.time import Time
import astropy.units as u

t = Time('2012-11-14T00:00:06.840')
get_pointing_table(t - 3*u.h, t + 3*u.h)

I'm not sure whether this is a JSOC issue (http://jsoc.stanford.edu/sdp_status/ seems to be fine), a drms issue, or if it's an error that aiapy should be catching and issuing a more user-freindly version.

I have

nabobalis commented 3 years ago

In GitLab by @wtbarnes on Nov 4, 2020, 10:45

Just playing around with this a bit, I can get it to not error by expanding the window to +/- 12 hours of your desired time. For some reason, it looks like there is only one pointing table entry available on 2012-11-13T18:00:00 and 2012-11:14T21:00:00 (see table below). Why that is the case, I have no idea, but we should probably have a way to more gracefully deal with this. In the actually pointing correction code, this won't matter is it will just find the entry that is closest to the Map obstime.

ORIGIN TELESCOP DATE T_START T_STOP T_HKVALS VERSION SC_Y_INRT_BIAS SC_Z_INRT_BIAS H_CAM1_IMSCALE H_CAM1_X0 H_CAM1_Y0 H_CAM1_INSTROT H_CAM2_IMSCALE H_CAM2_X0 H_CAM2_Y0 H_CAM2_INSTROT A_094_IMSCALE A_094_X0 A_094_Y0 A_094_INSTROT A_131_IMSCALE A_131_X0 A_131_Y0 A_131_INSTROT A_171_IMSCALE A_171_X0 A_171_Y0 A_171_INSTROT A_193_IMSCALE A_193_X0 A_193_Y0 A_193_INSTROT A_211_IMSCALE A_211_X0 A_211_Y0 A_211_INSTROT A_304_IMSCALE A_304_X0 A_304_Y0 A_304_INSTROT A_335_IMSCALE A_335_X0 A_335_Y0 A_335_INSTROT A_1600_IMSCALE A_1600_X0 A_1600_Y0 A_1600_INSTROT A_1700_IMSCALE A_1700_X0 A_1700_Y0 A_1700_INSTROT A_4500_IMSCALE A_4500_X0 A_4500_Y0 A_4500_INSTROT HMI_FSW_AL1_POSITION HMI_FSW_AL2_POSITION HMI_AL1_STATUS HMI_AL2_STATUS HMI_ISS_ERRGAINY HMI_ISS_ERRGAINZ HMI_ISS_ERROFFY HMI_ISS_ERROFFZ HMI_ISS_PZTOFFA HMI_ISS_PZTOFFB HMI_ISS_PZTOFFC HMI_ISS_PKT_YCOEF_A HMI_ISS_PKT_YCOEF_B HMI_ISS_PKT_YCOEF_C HMI_ISS_PKT_ZCOEF_A HMI_ISS_PKT_ZCOEF_B HMI_ISS_PKT_ZCOEF_C AIA_IS1_ERRGAINY AIA_IS1_ERRGAINZ AIA_IS1_ERROFFY AIA_IS1_ERROFFZ AIA_IS1_PZTGAINA AIA_IS1_PZTGAINB AIA_IS1_PZTGAINC AIA_IS1_PZTOFFA AIA_IS1_PZTOFFB AIA_IS1_PZTOFFC AIA_GT1_PKT_YCOEF_A AIA_GT1_PKT_YCOEF_B AIA_GT1_PKT_YCOEF_C AIA_GT1_PKT_ZCOEF_A AIA_GT1_PKT_ZCOEF_B AIA_GT1_PKT_ZCOEF_C AIA_IS2_ERRGAINY AIA_IS2_ERRGAINZ AIA_IS2_ERROFFY AIA_IS2_ERROFFZ AIA_IS2_PZTGAINA AIA_IS2_PZTGAINB AIA_IS2_PZTGAINC AIA_IS2_PZTOFFA AIA_IS2_PZTOFFB AIA_IS2_PZTOFFC AIA_GT2_PKT_YCOEF_A AIA_GT2_PKT_YCOEF_B AIA_GT2_PKT_YCOEF_C AIA_GT2_PKT_ZCOEF_A AIA_GT2_PKT_ZCOEF_B AIA_GT2_PKT_ZCOEF_C AIA_IS3_ERRGAINY AIA_IS3_ERRGAINZ AIA_IS3_ERROFFY AIA_IS3_ERROFFZ AIA_IS3_PZTGAINA AIA_IS3_PZTGAINB AIA_IS3_PZTGAINC AIA_IS3_PZTOFFA AIA_IS3_PZTOFFB AIA_IS3_PZTOFFC AIA_GT3_PKT_YCOEF_A AIA_GT3_PKT_YCOEF_B AIA_GT3_PKT_YCOEF_C AIA_GT3_PKT_ZCOEF_A AIA_GT3_PKT_ZCOEF_B AIA_GT3_PKT_ZCOEF_C AIA_IS4_ERRGAINY AIA_IS4_ERRGAINZ AIA_IS4_ERROFFY AIA_IS4_ERROFFZ AIA_IS4_PZTGAINA AIA_IS4_PZTGAINB AIA_IS4_PZTGAINC AIA_IS4_PZTOFFA AIA_IS4_PZTOFFB AIA_IS4_PZTOFFC AIA_GT4_PKT_YCOEF_A AIA_GT4_PKT_YCOEF_B AIA_GT4_PKT_YCOEF_C AIA_GT4_PKT_ZCOEF_A AIA_GT4_PKT_ZCOEF_B AIA_GT4_PKT_ZCOEF_C
arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg arcsec / pix arcsec arcsec deg
str12 str3 str20 object object str20 int64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64 float64
SDO/JSOC-SDP SDO 2017-10-24T16:52:21Z 2012-11-13T15:00:00.000 2012-11-13T18:00:00.000 2012-11-11T00:00:00Z 0 -2.519 9.982 0.50489 2036.969971 2047.3499760000002 180.013504 0.5048600000000001 2042.609985 2043.819946 179.929794 0.600109 2059.971191 2037.0517579999998 -0.13760999999999998 0.600698 2041.83667 2041.151611 -0.138795 0.599489 2052.252686 2046.4403079999997 0.019327 0.6007140000000001 2041.89917 2041.780029 0.057789 0.600758 2035.96228 2040.379761 0.056433000000000004 0.600165 2059.222168 2034.934814 -0.131639 0.600737 2040.36731 2038.862671 -0.14244300000000001 0.6093729999999999 2049.6567379999997 2045.856079 0.019791999999999997 0.6128979999999999 2051.102539 2044.183228 0.020525 0.599945 2049.600098 2047.6033940000002 0.020638 62.0 57.0 81.0 65.0 -43.0 -44.0 -37.0 66.0 -16.0 -16.0 -13.0 -20920.0 -10460.0 -10460.0 0.0 -15098.0 -15098.0 -85.0 -85.0 -17.0 53.0 -19.0 -15.0 -15.0 -15.0 -17.0 32.0 2048.0 13900.0 0.0 2048.0 13900.0 0.0 -85.0 -85.0 1.0 38.0 -14.0 -26.0 -26.0 11.0 5.0 -15.0 2048.0 13900.0 0.0 2048.0 13900.0 0.0 -79.0 -78.0 6.0 13.0 -12.0 -20.0 -20.0 -37.0 9.0 29.0 2048.0 13900.0 0.0 2048.0 13900.0 0.0 -80.0 -80.0 7.0 7.0 -14.0 -19.0 -19.0 25.0 1.0 -26.0 2048.0 13900.0 0.0 2048.0 13900.0 0.0
SDO/JSOC-SDP SDO 2017-10-24T16:52:21Z 2012-11-13T18:00:00.000 2012-11-14T21:00:00.000 2012-11-11T00:00:00Z 0 -2.519 9.982 0.50489 2036.969971
nabobalis commented 3 years ago

In GitLab by @dstansby1 on Nov 4, 2020, 10:59

Thanks for the quick reply! Looks like both

I'll open upstream issues in both sunpy and drms for the second one, and can open a PR here for the first one.

nabobalis commented 3 years ago

In GitLab by @wtbarnes on Nov 4, 2020, 15:11

I'm wondering if this will be affected by the metadata search refactor in sunpy: https://github.com/sunpy/sunpy/pull/4358.

nabobalis commented 3 years ago

In GitLab by @wtbarnes on Nov 8, 2020, 20:14

mentioned in commit 0ecf12ac7ebd268f0cf376d79b0e36443cc8ab82