BUG Problem with flatpix

[robertdstein]

Describe the bug I have discovered unexpected key errors in some cases, where pixels are not found. I already tried deleting the flatpix cache, which did not help.

To Reproduce Steps to reproduce the behavior:

1. Run skymap scan for "https://heasarc.gsfc.nasa.gov/FTP/fermi/data/gbm/triggers/2023/bn230518951/quicklook/glg_healpix_all_bn230518951.fit"
2. do plot_coverage()

It runs fine for ~24/577 pixels, and then:

Traceback:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[9], line 1
----> 1 scanner.plot_coverage()

File ~/Code/nuztf/nuztf/skymap_scanner.py:488, in SkymapScanner.plot_coverage(self, plot_candidates)
    486 def plot_coverage(self, plot_candidates: bool = True):
    487     """Plot ZTF coverage of skymap region"""
--> 488     fig, message = self.plot_overlap_with_observations(
    489         first_det_window_days=self.n_days
    490     )
    492     if plot_candidates:
    493         for candidate, res in self.cache.items():

File ~/Code/nuztf/nuztf/base_scanner.py:1168, in BaseScanner.plot_overlap_with_observations(self, first_det_window_days, min_sep)
   1159 def plot_overlap_with_observations(self, first_det_window_days=None, min_sep=0.01):
   1160     """
   1161     Function to plot the overlap of the field with observations.
   1162 
   (...)
   1165 
   1166     """
-> 1168     overlap_res = self.calculate_overlap_with_depot_observations(
   1169         first_det_window_days=first_det_window_days,
   1170         min_sep=min_sep,
   1171     )
   1172     if overlap_res is None:
   1173         self.logger.info("IPAC depot failed, using ztfquery to obtain observations")

File ~/Code/nuztf/nuztf/base_scanner.py:1043, in BaseScanner.calculate_overlap_with_depot_observations(self, first_det_window_days, min_sep)
   1039 obs = data[data["obsjd"] == obs_time]
   1041 field = obs["field_id"].iloc[0]
-> 1043 flat_pix = nested_pix[field]
   1045 mask = obs["status"] == 0
   1046 indices = obs["qid"].values[mask]

KeyError: 700685

Expected behaviour This should just work.

[robertdstein]

The answer: IN2P3 engineering/calibration observations! We can try/except around this.