Saving AAPP-processed NOAA HRPT to NinJoTIFF fails with AttributeError

[gerritholl]

Describe the bug

When loading a NOAA AVHRR HRPT Level-1B file produced with AAPP and saving it to NinJoTIFF, pint raises an exception AttributeError: 'NoneType' object has no attribute 'evaluate'.

To reproduce

from satpy import Scene
from satpy.utils import debug_on; debug_on()
sc = Scene(
        filenames=["/media/nas/x21308/scratch/hrpt_noaa19_20210324_0828_62464.l1b"],
        reader=["avhrr_l1b_aapp"])
sc.load(["1"])
ls = sc.resample("eurol")
ls.save_dataset("1",
        "/tmp/{platform_name}-{area.area_id}-{start_time:%Y%m%d_%H%M}-{end_time:%Y%m%d_%H%M}.tif",
        writer="ninjotiff",
        physic_value="ALBEDO",
        physic_unit="%",
        sat_id=7900014,
        chan_id=100015,
        data_cat="PORN",
        nbits=8,
        data_source="HRPT",
        ch_min_measurement_unit=0,
        ch_max_measurement_unit=125)

Expected behaviour

I expect that a NinJoTIFF file is written with the indicated information.

Actual results

[DEBUG: 2021-03-24 10:27:15 : satpy.readers.yaml_reader] Reading ('/home/gholl/checkouts/satpy/satpy/etc/readers/avhrr_l1b_aapp.yaml',)
[DEBUG: 2021-03-24 10:27:15 : satpy.readers.yaml_reader] Assigning to avhrr_l1b_aapp: ['/media/nas/x21308/scratch/hrpt_noaa19_20210324_0828_62464.l1b']
[DEBUG: 2021-03-24 10:27:15 : satpy.readers.aapp_l1b] Reading time 0:00:00.003219
[DEBUG: 2021-03-24 10:27:15 : satpy.composites.config_loader] Looking for composites config file avhrr-3.yaml
[DEBUG: 2021-03-24 10:27:15 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2021-03-24 10:27:15 : satpy.readers.yaml_reader] No coordinates found for DataID(name='longitude', resolution=1050, modifiers=())
[DEBUG: 2021-03-24 10:27:15 : satpy.readers.yaml_reader] No coordinates found for DataID(name='latitude', resolution=1050, modifiers=())
[INFO: 2021-03-24 10:27:15 : satpy.readers.aapp_l1b] No valid operational coefficients, fall back to pre-launch
[DEBUG: 2021-03-24 10:27:15 : pyproj] PROJ_ERROR: proj_crs_get_sub_crs: Object is not a CompoundCRS
[DEBUG: 2021-03-24 10:27:15 : pyproj] PROJ_ERROR: proj_crs_get_sub_crs: Object is not a CompoundCRS
[DEBUG: 2021-03-24 10:27:15 : satpy.scene] Resampling DataID(name='1', wavelength=WavelengthRange(min=0.58, central=0.63, max=0.68, unit='µm'), resolution=1050, calibration=<calibration.reflectance>, modifiers=())
[INFO: 2021-03-24 10:27:15 : satpy.scene] Not reducing data before resampling.
[INFO: 2021-03-24 10:27:15 : satpy.resample] Using default KDTree resampler
/data/gholl/miniconda3/envs/py39/lib/python3.9/site-packages/pyresample/geometry.py:2118: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  hor_xyz = np.stack(transform(src, dst, x, mid_row_y, alt_x), axis=1)
/data/gholl/miniconda3/envs/py39/lib/python3.9/site-packages/pyresample/geometry.py:2119: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  vert_xyz = np.stack(transform(src, dst, mid_col_x, y, alt_y), axis=1)
[DEBUG: 2021-03-24 10:27:15 : satpy.resample] Check if ./resample_lut-b444378bd3f422170b51d20660d8af7fb545fd3f.npz exists
[DEBUG: 2021-03-24 10:27:15 : satpy.resample] Computing kd-tree parameters
/data/gholl/miniconda3/envs/py39/lib/python3.9/site-packages/pyresample/kd_tree.py:1025: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  input_coords = input_coords.astype(np.float)
[DEBUG: 2021-03-24 10:27:15 : pyproj] PROJ_ERROR: proj_crs_get_sub_crs: Object is not a CompoundCRS
[DEBUG: 2021-03-24 10:27:15 : pyproj] PROJ_ERROR: proj_crs_get_sub_crs: Object is not a CompoundCRS
/data/gholl/miniconda3/envs/py39/lib/python3.9/site-packages/pyresample/kd_tree.py:1047: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  radius=self.radius_of_influence, dtype=np.int,
[DEBUG: 2021-03-24 10:27:15 : satpy.resample] Resampling where-60d4d27451e64f80225b160f9967589e
[DEBUG: 2021-03-24 10:27:15 : satpy.writers] Reading ['/home/gholl/checkouts/satpy/satpy/etc/writers/ninjotiff.yaml']
/data/gholl/miniconda3/envs/py39/lib/python3.9/site-packages/pyninjotiff/tifffile.py:154: UserWarning: failed to import the optional _tifffile C extension module.
Loading of some compressed images will be slow.
Tifffile.c can be obtained at http://www.lfd.uci.edu/~gohlke/
  warnings.warn(
Traceback (most recent call last):
  File "/home/gholl/checkouts/protocode/mwe/ninjotiff-crash.py", line 8, in <module>
    ls.save_dataset("1",
  File "/home/gholl/checkouts/satpy/satpy/scene.py", line 993, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "/home/gholl/checkouts/satpy/satpy/writers/ninjotiff.py", line 182, in save_dataset
    dataset = convert_units(dataset, units, nunits)
  File "/home/gholl/checkouts/satpy/satpy/writers/ninjotiff.py", line 99, in convert_units
    in_unit = ureg.parse_expression(in_unit, False)
  File "/data/gholl/miniconda3/envs/py39/lib/python3.9/site-packages/pint/registry.py", line 1263, in parse_expression
    return build_eval_tree(gen).evaluate(
AttributeError: 'NoneType' object has no attribute 'evaluate'

Environment info

Python 3.9.2, satpy 0.26, pyresample 0.18.1, pyninjotiff 0.3.0, pint 0.17.

Additional context

I'm not sure if this is a problem in Satpy, pyninjotiff, or pint, but I think pyninjotiff is the prime suspect.

Saving AAPP-processed Metop AVHRR files to NinJoTIFF works fine.

Unfortunately the files in question are larger than 10 MB, so I cannot upload them here.

[gerritholl]

This may be due to a bug in pint, see https://github.com/hgrecco/pint/issues/1274. I'm not sure why it affects processing NOAA but not Metop files.

[gerritholl]

Correction: it does also affect Metop files, but for some reason that code isn't reached for Metop in our trollflow2 production.

[mraspaud]

I think that bug appeared in newer versions of pint. This was flagged by @tecnavia-dev a while ago, but we never got around to fixing it.

After thinking about this for a while, I think pint is overkill here. The only thing we need to convert between for ninjo is celsius to kelvin or kelvin to celsius. So I think we should just do the conversion with + or - 273.15 ourselves. That would allow us to remove the pint requirement altogether.

[mraspaud]

(And I think this is done in satpy's ninjo writer)

[gerritholl]

On a closer look it's indeed in satpy.ninjotiff, should we move this issue to the satpy repository?

I don't know what the writer is trying to do here. It shouldn't be converting between celsius and kelvin for a reflectance channel!

[mraspaud]

yes, we should move it. No, the conversion should just be done for IR channels