undefined reference to `jpc_encode' && undefined reference to `jpc_decode'

[StevePny]

I have an error when building on aarch64 / arm64 / apple M1 in a docker container using either ubuntu or fedora:

[ 54%] Linking C executable wgrib2
/usr/bin/ld: CMakeFiles/obj_lib.dir/enc_jpeg2000_clone.c.o: in function `enc_jpeg2000_clone':
/tmp/NCEPLIBS-wgrib2/wgrib2/enc_jpeg2000_clone.c:174: undefined reference to `jpc_encode'
/usr/bin/ld: CMakeFiles/obj_lib.dir/unpk.c.o: in function `unpk_grib':
/tmp/NCEPLIBS-wgrib2/wgrib2/unpk.c:200: undefined reference to `jpc_decode'
collect2: error: ld returned 1 exit status
make[2]: *** [wgrib2/CMakeFiles/wgrib2_exe.dir/build.make:636: wgrib2/wgrib2] Error 1
make[1]: *** [CMakeFiles/Makefile2:183: wgrib2/CMakeFiles/wgrib2_exe.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

The include statement for jasper is here:

root@50e2e0eb2eee:/tmp/NCEPLIBS-wgrib2/build# vim /tmp/NCEPLIBS-wgrib2/wgrib2/enc_jpeg2000_clone.c

#include <stdio.h>
#include <stdlib.h>
#include "wgrib2.h"

#ifdef USE_JASPER
#include <jasper/jasper.h>
#define JAS_1_700_2

int enc_jpeg2000_clone(unsigned char *cin,int width,int height,int nbits,
                 int ltype, int ratio, int retry, char *outjpc,
                 int jpclen)

In jasper the jpc_encode and jpc_decode are in jasper/jas_image.h:

root@50e2e0eb2eee:/tmp/NCEPLIBS-wgrib2/build# grep encode /opt/jasper/include/jasper/*
/opt/jasper/include/jasper/jas_image.h: int (*encode)(jas_image_t *image, jas_stream_t *out, const char *opts);
/opt/jasper/include/jasper/jas_image.h:int jas_image_encode(jas_image_t *image, jas_stream_t *out, int fmt,
/opt/jasper/include/jasper/jas_image.h:int jpg_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:int jas_heic_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:int mif_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:int pnm_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:int ras_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:int bmp_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:int jp2_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:int jpc_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:int pgx_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);

/opt/jasper/include/jasper/jas_image.h: jas_image_t *(*decode)(jas_stream_t *in, const char *opts);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *jas_image_decode(jas_stream_t *in, int fmt, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *jpg_decode(jas_stream_t *in, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *jas_heic_decode(jas_stream_t *in, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *mif_decode(jas_stream_t *in, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *pnm_decode(jas_stream_t *in, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *ras_decode(jas_stream_t *in, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *bmp_decode(jas_stream_t *in, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *jp2_decode(jas_stream_t *in, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *jpc_decode(jas_stream_t *in, const char *optstr);
/opt/jasper/include/jasper/jas_image.h:jas_image_t *pgx_decode(jas_stream_t *in, const char *optstr);

But jasper/jasper.h includes jasper/jas_image.h:

root@50e2e0eb2eee:/tmp/NCEPLIBS-wgrib2/build# vim /opt/jasper/include/jasper/jasper.h

/* The configuration header file should be included first. */
#include <jasper/jas_config.h>

#include <jasper/jas_types.h>
#include <jasper/jas_version.h>
...
#include <jasper/jas_image.h>
...

In:

root@50e2e0eb2eee:/tmp/NCEPLIBS-wgrib2/build# vim /opt/jasper/include/jasper/jas_image.h 

It shows that these functions are only defined if JAS_INCLUDE_JPC_CODEC is defined:

#if defined(JAS_INCLUDE_JPC_CODEC)
/* Format-dependent operations for JPEG-2000 code stream support. */
//JAS_EXPORT
jas_image_t *jpc_decode(jas_stream_t *in, const char *optstr);
//JAS_EXPORT
int jpc_encode(jas_image_t *image, jas_stream_t *out, const char *optstr);
//JAS_EXPORT
int jpc_validate(jas_stream_t *in);
#endif

I commented out the "#if defined" statement above to ensure that this is included in the cmake build.

I tried adding these options when building Jasper, but it didn't help:

-DJAS_ENABLE_LIBJPEG=true -DJAS_INCLUDE_JPC_CODEC=true -DJAS_ENABLE_JPC_CODEC=true

So these should be visible, but they are not seen by the linker. I'm still not sure why.

I can compile if I set:

-DUSE_JASPER=OFF

[100%] Building C object aux_progs/CMakeFiles/gmerge.dir/uint8.c.o
cc1: warning: command line option '-fbacktrace' is valid for Fortran but not for C
[100%] Linking C executable gmerge
[100%] Built target gmerge
Scanning dependencies of target wgrib2_api
[100%] Building Fortran object ftn_api/CMakeFiles/wgrib2_api.dir/wgrib2lowapi.f90.o
[100%] Building Fortran object ftn_api/CMakeFiles/wgrib2_api.dir/wgrib2api.f90.o
[100%] Building C object ftn_api/CMakeFiles/wgrib2_api.dir/fort_wgrib2.c.o
cc1: warning: command line option '-fbacktrace' is valid for Fortran but not for C
[100%] Linking Fortran static library libwgrib2_api.a
[100%] Built target wgrib2_api

[kgerheiser]

This repo is a fork of the official repo with a CMake build based on wgrib2 2.0.8.

We are no longer maintaining this fork because of difficulties keeping up-to-date with the upstream version.

I would recommend using the official repo at: https://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/ (download here https://www.ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/)

[StevePny]

@kgerheiser This is the repo for wgrib2 that is pointed to by https://github.com/NOAA-EMC/NCEPLIBS-external . Could you coordinate with them to update the links in their repo if this is out of date, since that is what is being suggested for all UFS community builds as far as I can tell? As it currently stands, wgrib2 breaks the entire NCEPLIBS-external build for me. I'm working on building wgrib2 separately in a Docker container and shutting it off in the NCEPLIB-external cmake build, but it should be fixed there.

I've tried v3.1.0 and it doesn't build for me in ubuntu: root@9ef4a95fb85f:/tmp/grib2# make cd /tmp/grib2/zlib-1.2.11 && export CFLAGS="-I/opt/hdf5/include -I/tmp/grib2/include -O2 -DIFORT -cxxlib -qopenmp -I/tmp/grib2/jasper-1.900.1/src/libjasper/include " && ./configure --prefix=/tmp/grib2 --static && make install Compiler error reporting is too harsh for ./configure (perhaps remove -Werror). ** ./configure aborting. make: *** [makefile:903: /tmp/grib2/lib/libz.a] Error 1

The best I can do right now to get a successful build is to turn off Jasper: RUN cd /tmp \ && git clone https://github.com/NOAA-EMC/NCEPLIBS-wgrib2 \ && cd NCEPLIBS-wgrib2 \ && git checkout ${WGRIB2_VERSION} \ && mkdir -p build && cd build \ && cmake -DCMAKE_INSTALL_PREFIX='/opt' -DUSE_JASPER=OFF .. \ && make -j${MAKEJOBS} && make install

[edwardhartnett]

We will remove wgrib2 from NCEPLIBS-external.

[StevePny]

I found a solution to this problem. Apparently, this repo may be using features that are no longer supported by the more recent versions of Jasper. I changed from Jasper version 3.0.2 to Jasper version 2.0.33 and all of wgrib2, nceplibs, and ufstools build in the ubuntu-arm64 environment. In this case, I have to build jasper from source:

RUN apt-get update && \
    apt-get -y install \
    libpng-dev \
    libjpeg-dev

RUN cd /tmp \
  && git clone https://github.com/jasper-software/jasper.git \
  && cd jasper \
  && git checkout version-${JASPER_VERSION} \
  && mkdir -p build \
  && cmake -H./ -B./build -DCMAKE_INSTALL_PREFIX=$JASPER_DIR  ${JASPER_OPTIONS} \
  && cmake --build build \
  && cmake --build build --target install
ENV Jasper_ROOT=$JASPER_DIR

ENV LD_LIBRARY_PATH=${JASPER_DIR}/lib:${LD_LIBRARY_PATH}

[kgerheiser]

That seems plausible. We have never tested with Jasper 3.x.

[edwardhartnett]

Well looks like we need to upgrade jasper...

[edwardhartnett]

I upgraded jasper and am seeing this problem with NCEPLIBS-g2.

[t-brown]

Just hit this issue too. I'm going to pin the Spack package to v2. PR is 30736

[kgerheiser]

FYI @t-brown we've got a Spack build of wgrib2 that uses the official source, so you can use new versions. The CMake build isn't being updated anymore. I think we're planning to merge this into the main Spack repo, if you have any thoughts. On the upside you get new versions, but on the downside you have to deal with wgrib2's quirkiness in packaging ancient internal libraries (but I think we've fixed that with various patches).

https://github.com/NOAA-EMC/spack/blob/jcsda_emc_spack_stack/var/spack/repos/builtin/packages/wgrib2/package.py

[t-brown]

@kgerheiser Yes, I'd love to see this merged into the main Spack repo! Only one very small nitpick. On line 31, s/grib/GRIB/ as it is an acroynm?

[bilias]

I've managed to compile (WPS/ungrib.exe) by forcing the static jasper lib. in configure.wps

COMPRESSION_LIBS = -L/usr/local/apps/jasper/3.0.5/intel/lib64 -ljpeg -ljasper -lpng -lz

COMPRESSION_LIBS = /usr/local/apps/jasper/3.0.5/intel/lib64/libjasper.a -ljpeg -lpng -lpthread

ungrib/src/ngl/g2/Makefile says: # -DUSE_JPEG2000 requires libjasper.a

[edwardhartnett]

wgrib2 is now being maintained with the CMake build, and the GitHub site is the sole home of wgrib2. All future releases will have a CMake build system.

Some updates:

• the NCEPLIBS-g2c library now handles all versions of Jasper.
• The next release (wgrib2 -3.3.0) will still require Jasper-2.x.
• The following release, wgrib2-3.4.0, will support multiple Jasper versions by calling the NCEPLIBS-g2c library.