eleanor.TargetData fails with example

[jvines]

Hi,

Following the example in the documentation I ran:

star = eleanor.Source(tic=38846515, sector=1)
data = eleanor.TargetData(star, height=15, width=15, bkg_size=31, do_psf=True, do_pca=True)

And got a LinAlgError alongside some warnings. The warnings I think are harmless, but for completion I'll add them as well:

Warnings:

WARNING: FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 58324.812768 from DATE-OBS.
Set DATE-BEG to '2018-07-25T19:31:32.353' from MJD-BEG'. [astropy.wcs.wcs]
WARNING: The "exact" method is not yet implemented for rectangular apertures -- using "subpixel" method with "subpixels=32" [photutils.aperture.core]

The error traceback is:

-------------------------------------
LinAlgErrorTraceback (most recent call last)
<ipython-input-12-07bdbfe9cc9a> in <module>
----> 1 data = eleanor.TargetData(star, height=15, width=15, bkg_size=31, do_psf=True, do_pca=True)
      2 
      3 q = data.quality == 0

/anaconda3/lib/python3.7/site-packages/eleanor-1.0.3-py3.7.egg/eleanor/targetdata.py in __init__(self, source, height, width, save_postcard, do_pca, do_psf, bkg_size, crowded_field, cal_cadences, try_load, language)
    211 
    212                 if do_pca == True:
--> 213                     self.corrected_flux(pca=True)
    214                 else:
    215                     self.modes = None

/anaconda3/lib/python3.7/site-packages/eleanor-1.0.3-py3.7.egg/eleanor/targetdata.py in corrected_flux(self, flux, skip, modes, pca, bkg)
   1211         f   = np.arange(0, brk, 1); s = np.arange(brk, len(self.time), 1)
   1212 
-> 1213         lc_pred = calc_corr(f, cx, cy, skip)
   1214 
   1215         corr_f = flux[f]/lc_pred * med

/anaconda3/lib/python3.7/site-packages/eleanor-1.0.3-py3.7.egg/eleanor/targetdata.py in calc_corr(mask, cx, cy, skip)
   1202 
   1203 
-> 1204             x = xhat(cm, norm_l[skip:])
   1205             fmod = fhat(x, cm_full)
   1206             lc_pred = (fmod+1)

/anaconda3/lib/python3.7/site-packages/eleanor-1.0.3-py3.7.egg/eleanor/targetdata.py in xhat(mat, lc)
   1143         def xhat(mat, lc):
   1144             ATA = np.dot(mat.T, mat)
-> 1145             ATAinv = np.linalg.inv(ATA)
   1146             ATf = np.dot(mat.T, lc)
   1147             xhat = np.dot(ATAinv, ATf)

<__array_function__ internals> in inv(*args, **kwargs)

/anaconda3/lib/python3.7/site-packages/numpy/linalg/linalg.py in inv(a)
    549     signature = 'D->D' if isComplexType(t) else 'd->d'
    550     extobj = get_linalg_error_extobj(_raise_linalgerror_singular)
--> 551     ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj)
    552     return wrap(ainv.astype(result_t, copy=False))
    553 

/anaconda3/lib/python3.7/site-packages/numpy/linalg/linalg.py in _raise_linalgerror_singular(err, flag)
     95 
     96 def _raise_linalgerror_singular(err, flag):
---> 97     raise LinAlgError("Singular matrix")
     98 
     99 def _raise_linalgerror_nonposdef(err, flag):

LinAlgError: Singular matrix

I don't know what might be causing this as I have ran the tutorial in the past and it has worked. I tried varying the height, width and bkg_size values to no avail, so any help would be appreciated!

Thank you!

[benmontet]

Oh this is #142 again I think. We need to put in an error message to better handle this one.

Can you please remove your ~/.eleanor/metadata directory and try again? It downloads some data behind the scenes to use for the corrections and in this case I think it only got some of them, but thinks all are there and is getting confused and failing.

[jvines]

OK. I did that and now I get

-------------------------------------
AttributeErrorTraceback (most recent call last)
<ipython-input-5-07bdbfe9cc9a> in <module>
----> 1 data = eleanor.TargetData(star, height=15, width=15, bkg_size=31, do_psf=True, do_pca=True)
      2 
      3 q = data.quality == 0

/anaconda3/lib/python3.7/site-packages/eleanor-1.0.3-py3.7.egg/eleanor/targetdata.py in __init__(self, source, height, width, save_postcard, do_pca, do_psf, bkg_size, crowded_field, cal_cadences, try_load, language)
    223                 self.center_of_mass()
    224 
--> 225                 self.set_header()
    226 
    227 

/anaconda3/lib/python3.7/site-packages/eleanor-1.0.3-py3.7.egg/eleanor/targetdata.py in set_header(self)
   1280         self.header.append(fits.Card(keyword='BKG_LVL', value=self.bkg_type,
   1281                                      comment='Stage at which background is subtracted'))
-> 1282         self.header.append(fits.Card(keyword='URL', value=self.source_info.ELEANORURL,
   1283                                      comment='URL eleanor files are located at'))
   1284 

AttributeError: 'Source' object has no attribute 'ELEANORURL'

:(

[benmontet]

I'm going to defer to @afeinstein20 here, because I think she was doing some cleaning up in those functions.

Adina, it looks like self.eleanorurl is presently commented out, but is still trying to be recorded in the header. Now that everything is from mast, how should we handle what's recorded here in the FITS header? Maybe just whether it was made with local postcards or ones from MAST, and if MAST then also store self.postcard_path?

Jose, for right now can you please go back one commit ("git checkout 509f90ca8e33d02c4d0db068fc640462eba937ba") and verify that it works on your end from that version?

(Note github seems to be shortening the hash here, at least on my view. If you click on it you can get the full hash.)

[jvines]

@benmontet I reverted back to that hash value (you don't need the full hash for the checkout so everything is OK) and it worked!

Thanks for the help :) I'll keep an eye out for when the issue is fully solved 👍

[benmontet]

Perfect! I'll leave this open as a reminder that we need to sort out that header issue. Good luck, and let us know if you run into anything else strange!

[benmontet]

Okay I changed around what's stored in the header so this should be sorted moving forward, you should be able to use the latest version from here!