libpng error: bad parameters to zlib

[davetapley]

Issue:

$ wgrib2 file.grib2 -netcdf out.nc
libpng error: bad parameters to zlib

*** FATAL ERROR: unpk: png decode error -3 ***

Encountered with .grib2 files found here.

Environment (conda list):

``` $ conda list # packages in environment at /root/miniconda3/envs/temp: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.17.1 h36c2ea0_0 conda-forge ca-certificates 2020.11.8 ha878542_0 conda-forge curl 7.71.1 he644dc0_8 conda-forge g2clib 1.6.0 hf3f1b0b_9 conda-forge hdf4 4.2.13 h10796ff_1004 conda-forge hdf5 1.10.6 nompi_h6a2412b_1112 conda-forge jasper 1.900.1 h07fcdf6_1006 conda-forge jpeg 9d h36c2ea0_0 conda-forge krb5 1.17.2 h926e7f8_0 conda-forge libcurl 7.71.1 hcdd3856_8 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libgcc-ng 9.3.0 h5dbcf3e_17 conda-forge libgfortran-ng 9.3.0 he4bcb1c_17 conda-forge libgfortran5 9.3.0 he4bcb1c_17 conda-forge libgomp 9.3.0 h5dbcf3e_17 conda-forge libnetcdf 4.7.4 nompi_hefab0ff_106 conda-forge libnghttp2 1.41.0 h8cfc5f6_2 conda-forge libpng 1.6.37 h21135ba_2 conda-forge libssh2 1.9.0 hab1572f_5 conda-forge libstdcxx-ng 9.3.0 h2ae2ef3_17 conda-forge mysql-connector-c 6.1.11 h6eb9d5d_1007 conda-forge ncurses 6.2 h58526e2_4 conda-forge openssl 1.1.1h h516909a_0 conda-forge tk 8.6.10 hed695b0_1 conda-forge wgrib2 2.0.8 h5a10b45_1 conda-forge zlib 1.2.11 h516909a_1010 conda-forge ```

Details about conda and system ( conda info ):

``` $ conda info active environment : temp active env location : /root/miniconda3/envs/temp shell level : 3 user config file : /root/.condarc populated config files : /root/.condarc conda version : 4.9.2 conda-build version : not installed python version : 3.8.5.final.0 virtual packages : __glibc=2.23=0 __unix=0=0 __archspec=1=x86_64 base environment : /root/miniconda3 (writable) channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /root/miniconda3/pkgs /root/.conda/pkgs envs directories : /root/miniconda3/envs /root/.conda/envs platform : linux-64 user-agent : conda/4.9.2 requests/2.24.0 CPython/3.8.5 Linux/5.4.0-52-generic ubuntu/16.04.7 glibc/2.23 UID:GID : 0:0 netrc file : None offline mode : False ```

[davetapley]

I am seeing this on both xenial and bionic, but bizarrely it's working just fine (with the same files) on a production xenial machine (although I can't guarantee how that machine was configured).

The only notable difference is that it's using an older conda version: 4.3.30 vs 4.9.2.

Environment (conda list):

``` $ conda list # packages in environment at /home/alert/.conda/envs/radar: # asn1crypto 0.22.0 py36_0 conda-forge backports 1.0 py36_1 conda-forge backports.functools_lru_cache 1.5 py36_0 conda-forge biggus 0.15.0 py36_0 conda-forge boost 1.66.0 py36_1 conda-forge boost-cpp 1.66.0 1 conda-forge bzip2 1.0.6 1 conda-forge ca-certificates 2018.1.18 0 conda-forge cairo 1.14.10 0 conda-forge cartopy 0.15.1 py36_5 conda-forge cdo 1.3.5 cdo 1.9.2 0 conda-forge certifi 2018.1.18 py36_0 conda-forge cf_units 1.2.0 py36_0 conda-forge cffi 1.11.2 py36_0 conda-forge chardet 3.0.4 py36_0 conda-forge cryptography 2.1.4 py36_0 conda-forge curl 7.55.1 0 conda-forge cycler 0.10.0 py36_0 conda-forge dbus 1.10.22 0 conda-forge eccodes 2.6.0 1 conda-forge ecmwf-grib-api 1.23.1 0 eumetsat ecmwf_grib 1.23.1 2 conda-forge expat 2.2.5 0 conda-forge fftw 3.3.7 0 conda-forge fontconfig 2.12.4 h88586e7_1 freetype 2.8.1 0 conda-forge freexl 1.0.4 0 conda-forge g2clib 1.6.0 5 conda-forge gdal 2.2.3 py36_0 conda-forge geos 3.6.2 1 conda-forge geotiff 1.4.2 1 conda-forge gettext 0.19.8.1 0 conda-forge giflib 5.1.4 0 conda-forge glib 2.55.0 0 conda-forge gst-plugins-base 1.8.0 0 conda-forge gstreamer 1.8.0 1 conda-forge hdf4 4.2.13 h3ca952b_2 hdf5 1.10.1 1 conda-forge icu 58.2 0 conda-forge idna 2.6 py36_1 conda-forge intel-openmp 2018.0.0 hc7b2577_8 iris 1.13.0 py36_1 conda-forge jasper 1.900.1 4 conda-forge jpeg 9b 2 conda-forge json-c 0.12.1 0 conda-forge kealib 1.4.7 4 conda-forge krb5 1.14.2 0 conda-forge libdap4 3.18.3 2 conda-forge libffi 3.2.1 3 conda-forge libgcc 7.2.0 h69d50b8_2 libgcc-ng 7.2.0 h7cc24e2_2 libgdal 2.2.3 3 conda-forge libgfortran 3.0.0 1 libgfortran-ng 7.2.0 h9f7466a_2 libiconv 1.15 0 conda-forge libkml 1.3.0 6 conda-forge libmo_unpack 3.0 1 conda-forge libnetcdf 4.4.1.1 10 conda-forge libpng 1.6.34 hb9fc6fc_0 libpq 9.6.3 0 conda-forge libspatialite 4.3.0a 19 conda-forge libssh2 1.8.0 2 conda-forge libstdcxx-ng 7.2.0 h7a57d05_2 libtiff 4.0.9 h28f6b97_0 libxcb 1.12 1 conda-forge libxml2 2.9.7 0 conda-forge libxslt 1.1.32 0 conda-forge lxml 4.1.1 py36_0 conda-forge matplotlib 2.1.2 py36_0 conda-forge mkl 2018.0.1 h19d6760_4 mo_pack 0.2.0 py36_1 conda-forge mysql-connector-c 6.1.6 0 conda-forge nc_time_axis 1.0.2 py36_0 conda-forge ncurses 5.9 10 conda-forge netcdf4 1.3.1 py36hfd655bd_2 anaconda numpy 1.14.0 py36h3dfced4_1 olefile 0.45.1 py36_0 openjpeg 2.3.0 2 conda-forge openssl 1.0.2n 0 conda-forge owslib 0.16.0 py_0 conda-forge pcre 8.39 0 conda-forge pillow 5.0.0 py36h3deb7b8_0 pip 9.0.1 py36_1 conda-forge pixman 0.34.0 1 conda-forge poppler 0.61.1 3 conda-forge poppler-data 0.4.8 0 conda-forge proj4 4.9.3 5 conda-forge pycparser 2.18 py36_0 conda-forge pyepsg 0.3.2 py36_0 conda-forge pygrib 2.0.2 py36_3 conda-forge pyhdf 0.9.0 py36_2 conda-forge pyke 1.1.1 py36_0 conda-forge pyopenssl 17.4.0 py36_0 conda-forge pyparsing 2.2.0 py36_0 conda-forge pyproj 1.9.5.1 py36_0 conda-forge pyqt 5.6.0 py36_4 conda-forge pyshp 1.2.12 py_0 conda-forge pysocks 1.6.8 py36_1 conda-forge python 3.6.4 0 conda-forge python-dateutil 2.6.1 py36_0 conda-forge pytz 2017.3 py36h63b9c63_0 qt 5.6.2 7 conda-forge readline 7.0 0 conda-forge requests 2.18.4 py36_1 conda-forge scipy 1.0.0 py36hbf646e7_0 setuptools 38.5.1 py36_0 conda-forge shapely 1.6.4 py36_0 conda-forge sip 4.18 py36_1 conda-forge six 1.11.0 py36_1 conda-forge sqlite 3.20.1 2 conda-forge tk 8.6.7 0 conda-forge tornado 4.5.3 py36_0 conda-forge udunits2 2.2.25 2 conda-forge urllib3 1.22 py36_0 conda-forge util-linux 2.21 0 wgrib2 2.0.5 1 conda-forge wheel 0.30.0 py36_2 conda-forge xerces-c 3.2.0 0 conda-forge xorg-libxau 1.0.8 3 conda-forge xorg-libxdmcp 1.1.2 3 conda-forge xz 5.2.3 0 conda-forge zlib 1.2.11 0 conda-forge ```

Details about conda and system ( conda info ):

``` $ conda info Current conda install: platform : linux-64 conda version : 4.3.30 conda is private : False conda-env version : 4.3.30 conda-build version : 3.0.27 python version : 3.6.3.final.0 requests version : 2.18.4 root environment : /home/alert/anaconda3 (writable) default environment : /home/alert/.conda/envs/radar envs directories : /home/alert/anaconda3/envs /home/alert/.conda/envs package cache : /home/alert/anaconda3/pkgs /home/alert/.conda/pkgs channel URLs : https://repo.continuum.io/pkgs/main/linux-64 https://repo.continuum.io/pkgs/main/noarch https://repo.continuum.io/pkgs/free/linux-64 https://repo.continuum.io/pkgs/free/noarch https://repo.continuum.io/pkgs/r/linux-64 https://repo.continuum.io/pkgs/r/noarch https://repo.continuum.io/pkgs/pro/linux-64 https://repo.continuum.io/pkgs/pro/noarch config file : None netrc file : None offline mode : False user-agent : conda/4.3.30 requests/2.18.4 CPython/3.6.3 Linux/4.15.0-107-generic debian/stretch/sid glibc/2.23 UID:GID : 1000:1000 ```

[davetapley]

I was able to reproduce the error on with conda version 4.3.0.

If I copy the entire env dir from the machine where it works to a new machine, it works 🤪

diff on the wgrib2 binary confirms they differ, even though thought they should be identical.

The working version was created Feb 13 2018, so sometime between then and now it's become broken 😞

[davetapley]

Workaround

After starting with the working env dir, and conda uninstalling backwards, I was able to produce this minimal set of dependencies which do work:

dependencies:
- python=3.6.4=0
- pyhdf=0.9.0
- numpy=1.14.0=py36h3dfced4_1
- pillow=5.0.0=py36h3deb7b8_0
- pytz=2017.3=py36h63b9c63_0
- scipy=1.0.0=py36hbf646e7_0
- wgrib2=2.0.5=1
- jasper=1.900.1=4
- libpng=1.6.34=hb9fc6fc_0
- zlib=1.2.11=0
- g2clib=1.6.0=5
- mysql-connector-c=6.1.6=0
- libnetcdf=4.4.1.1=10
- libgcc=7.2.0=h69d50b8_2
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran=3.0.0=1
- libgfortran-ng=7.2.0=h9f7466a_2

[ocefpaf]

Thanks for the detailed issue!

We have three build numbers for version 2.0.5 and it is odd that only 1 is working since not much changed in them.

Do you know what is preventing you from getting a newer version of wgrib2 in that environment? We also have 2.0.7 and 2.0.8. Either way, we can remove the builds 0 and 2 of 2.0.5. I just want to add a test to the feedstock first to avoid regressions in future updates.

[davetapley]

@ocefpaf unfortunately it's some old code which has some monkey patches. I was hoping to get a set of dependencies first, then start working forwards.

Turns out there are other dependencies for my this old code which are also missing and/or broken in forge, so it looks like I'm going to be starting from scratch anyway.

At least I caught this on the way 🤪

[ocefpaf]

Thanks. I tried a few tests and combination of dependencies but nothing worked :-/ Let me know if you figure this one out.

[akrherz]

I engaged the wgrib2 developer on this and the current indication is that only libpng=1.2 is supported. Digging some more at this yet...

[akrherz]

This issue no longer seems reproducible.