Feature request: Overlap Area Crop just like pygac-fdr

[yukaribbba]

I need it in satpy. Thanks!

[sfinkens]

@yukaribbba You can read Pygac FDR files using the avhrr_l1c_eum_gac_fdr_nc reader and then use the overlap_free_start/end datasets to crop the overlap.

from satpy import Scene

filename = "AVHRR-GAC_FDR_1C_N06_19810330T005421Z_19810330T024632Z_R_O_20200101T000000Z_0100.nc"

scn = Scene([filename], reader="avhrr_l1c_eum_gac_fdr_nc")
scn.load(["reflectance_channel_1", "overlap_free_start", "overlap_free_end"])

refl = scn["reflectance_channel_1"]
start = scn["overlap_free_start"].astype(int).item()
end = scn["overlap_free_end"].astype(int).item()

overlap_free = refl.isel(y=slice(start, end))

[yukaribbba]

@sfinkens That's just what I need. But when I tried to produce a FDR dataset from a GAC file, an error raised:

python pygac-fdr-run --debug --cfg pygac-fdr.yaml --tle-dir D:\noaa_tles NSS.GHRR.TN.D79285.S0006.E0126.B0512728.gc

[INFO: 2022-10-11 17:37:03 : pygac_fdr] Processing file C:\Users\45107\Downloads\Sat\Polar\NSS.GHRR.TN.D79285.S0006.E0126.B0512728.gc
C:\users\45107\miniconda3\lib\site-packages\pygac\reader.py:274: RuntimeWarning: Unexpected number of scanlines!
  warnings.warn("Unexpected number of scanlines!",
C:\users\45107\miniconda3\lib\site-packages\pygac\calibration.py:198: RuntimeWarning: Using CoeffStatus.PROVISIONAL calibration coefficients
  warnings.warn(warning, RuntimeWarning)
C:\users\45107\miniconda3\lib\site-packages\geotiepoints\geointerpolator.py:81: RuntimeWarning: invalid value encountered in arcsin
  lats = np.sign(z__) * (90 - np.rad2deg(np.arcsin(np.sqrt(x__ ** 2 + y__ ** 2) / radius)))
C:\users\45107\miniconda3\lib\site-packages\pygac\calibration.py:544: RuntimeWarning: invalid value encountered in log
  tsE = c2*nu_c / np.log(1.0 + nBB_num / Ne)
C:\users\45107\miniconda3\lib\site-packages\pygac_fdr\writer.py:363: UserWarning: Attribute access to DataIDs is deprecated, use key access instead.
  scn_keys = set([key.name for key in scene.keys()])
[INFO: 2022-10-11 17:37:08 : pygac_fdr] Writing calibrated scene to C:\Users\45107\Downloads\Sat\Polar\AVHRR-GAC_FDR_1C_TSN_19791012T000633Z_19791012T012633Z_R_O_20200101T000000Z_0100.nc
C:\users\45107\miniconda3\lib\site-packages\satpy\writers\cf_writer.py:571: FutureWarning: The default behaviour of the CF writer will soon change to not compress data by default.
  warnings.warn("The default behaviour of the CF writer will soon change to not compress data by default.",
Traceback (most recent call last):
  File "C:\Users\45107\miniconda3\Scripts\pygac-fdr-run", line 134, in <module>
    process_file(filename, config)
  File "C:\Users\45107\miniconda3\Scripts\pygac-fdr-run", line 59, in process_file
    writer.write(scene=scene)
  File "C:\users\45107\miniconda3\lib\site-packages\pygac_fdr\writer.py", line 410, in write
    self._save_datasets(scene, filename, global_attrs)
  File "C:\users\45107\miniconda3\lib\site-packages\pygac_fdr\writer.py", line 429, in _save_datasets
    scene.save_datasets(
  File "C:\users\45107\miniconda3\lib\site-packages\satpy\scene.py", line 1159, in save_datasets
    return writer.save_datasets(dataarrays, compute=compute, **save_kwargs)
  File "C:\users\45107\miniconda3\lib\site-packages\satpy\writers\cf_writer.py", line 844, in save_datasets
    datas, start_times, end_times = self._collect_datasets(
  File "C:\users\45107\miniconda3\lib\site-packages\satpy\writers\cf_writer.py", line 764, in _collect_datasets
    assert_xy_unique(datas)
  File "C:\users\45107\miniconda3\lib\site-packages\satpy\writers\cf_writer.py", line 282, in assert_xy_unique
    raise ValueError('Datasets to be saved in one file (or one group) must have identical projection coordinates. '
ValueError: Datasets to be saved in one file (or one group) must have identical projection coordinates. Please group them by area or save them in separate files.

I'm using satpy 0.36 from pypi

[sfinkens]

@yukaribbba I tried with the same TIROS-N file but couldn't reproduce the error. However, I ran into another problem, which I fixed here: https://github.com/pytroll/pygac-fdr/pull/107. So please try upgrading to the latest version from github

pip install git+https://github.com/pytroll/pygac-fdr --upgrade --no-deps

Then you should be able to process that file.

[sfinkens]

I hope this was closed by https://github.com/pytroll/pygac-fdr/pull/107. Feel free to re-open