search

cedadev / jasmin_scivm

9 stars 0 forks source link

add mo_pack #76

Closed alaniwi closed 7 years ago

alaniwi commented 8 years ago

User-reported. The user writes (in FP 39310):

In the past week I have been using Python2.7 and some Met Office modules (Iris) on JASMIN to look at UM model output. The files are extracted from MASS in the Met Office PP format. For some variables (seems to be those related to land only) I get the following warning message:

/usr/lib/python2.7/site-packages/iris/fileformats/pp.py:913: UserWarning: iris.fileformats.pp_packing has been deprecated and will be removed in a future release. Install mo_pack to make use of the new unpacking functionality.

My scripts are also very slow when I get this warning message.

I have manged to install and link to mo_pack in my virtual environment which solves the problem (scripts now running as fast for land and non-land variables).

Perhaps this package could be added globally for other users that may encounter this issue.

Labelling as bug rather than enhancement in view of the deprecation warning.

alaniwi commented 7 years ago

Requires libmo_unpack (https://github.com/SciTools/libmo_unpack)

alaniwi commented 7 years ago

mo_unpack RPM already exists, but rebuilt using this much later version. Will require iris to be rebuilt (because .so version has changed).

alaniwi commented 7 years ago

Clean build of python27 package, and clean import mo_pack.

Quick test with some gibberish:

>>> mo_pack.decompress_wgdos("adasdfasdfsda", 2, 3)
size inconsistent 29540 * 25702 != 6
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lib/mo_pack/_packing.pyx", line 143, in mo_pack._packing.decompress_wgdos (lib/mo_pack/_packing.c:2204)
ValueError: WGDOS exit code was non-zero: -1

so, fails as expected, but looks like it is actually calling the C code as the "size inconsistent" message is coming from the C code - https://github.com/SciTools/libmo_unpack/search?utf8=%E2%9C%93&q=size+inconsistent

alaniwi commented 7 years ago

Updated Iris to 1.10.0 and compiled against latest mo_unpack. Now a new deprecation warning:

>>> import iris
/usr/lib/python2.7/site-packages/iris/fileformats/grib/__init__.py:59: IrisDeprecation: The module iris.fileformats.grib is deprecated since v1.10. Please install the package 'iris_grib' package instead.
  "The module iris.fileformats.grib is deprecated since v1.10. "
alaniwi commented 7 years ago

Built iris-grib and the deprecation warning goes away on importing iris. Rebuilt iris with extra dependency on iris-grib.