Related bug with saturated frames == 0

[ethankruse]

https://github.com/afeinstein20/eleanor/blob/1de8bdd2c82c2b995ce338ec854e886c197dae6e/eleanor/targetdata.py#L663-L672

Ok, so we're successfully making all postcards now, but the saturated frames now being set to 0 flux are causing more problems.

In this section, things are breaking because np.nanmedian(all_raw_lc_pc_sub[a]) is now 0. This is a peculiar case, and I can see why it's a rare occurrence. In this particular case, we're getting

>> (all_raw_lc_pc_sub[a]< 0).sum()
595
>> (all_raw_lc_pc_sub[a] > 0).sum()
596
>> (all_raw_lc_pc_sub[a] == 0).sum()
5

which of course means the median is 0, and you're then dividing by 0.

Now, my question is, what is this part of the code intending to do and is np.nanmedian(all_raw_lc_pc_sub[a]) supposed to be near 0 or even negative?

If I run this same star using your MAST postcards (TIC 271900883, sector 5) and breaking at aperture a == 8 which is where we're having the problem, I notice 2 things.

1. Where the last 5 cadences of all_raw_lc_tpf_sub[a] for us are 0, for you it's 88.9925036 , 86.8305255 , 100.92520905, 107.54550171, 35.41360568
2. More of your fluxes are negative than positive, which yields a negative median.

   >> (all_raw_lc_pc_sub[a] < 0).sum()
   649
   >> (all_raw_lc_pc_sub[a] > 0).sum()
   547
   >> (all_raw_lc_pc_sub[a] == 0).sum()
   0
   >>  np.nanmedian(all_raw_lc_pc_sub[a])
   -0.4104194641113281

Is that ok in general? And is the solution here just to once again remove all 0 cadences and proceed as usual?

[benmontet]

Okay interesting! This is an intriguing one.

Given perfect data with no noise or systematics, all_raw_lc_pc_sub[a] should be always positive. That variable is the flux time series performing simple aperture photometry over the ath aperture, with the background subtracted using a simple background model over the entire postcard (compare against all_raw_lc_tpf_sub which performs a tpf-level correction that usually but not always does better). So given you should have a star in your aperture, that should in principle always be positive!

That it's not either means the star is super faint or the background estimation is going wrong for some reason (a much brighter star nearby? A long streak from a saturated star? That sector 5 is weird generally?), or both of these. In principle you don't need to have that division in there, but I seem to remember the correction step, which is where self.corrected_flux is taking you, is more stable when the flux is approximately 1-centered rather than some arbitrary flux value so the median division is just there to get you to that point. You definitely don't want it to be zero, because that will crash, and negative will flip the light curve and it will never get un-flipped which is also bad. I think it should be safe to consider only the positive cadences in this step and run with that, it won't affect the morphology of the light curve you end up with, if it changes anything would be a multiplicative scale factor onto all the fluxes, which is fine because almost no one will care about trying to convert TESS electrons/second into a magnitude anyway (I really hope, at least!)

This is hopefully a very rare problem, since it's saying the median brightness of your star in the data for this sector is negative!

[benmontet]

This is sorted with #244, yes? I believe the answer is yes and am going to close it; if I'm wrong please tell me so and re-open it!