upgrade libhkl

[prjemian]

The libhkl version available on conda forge is now old (5.0.0.2173). It needs an update (5.0.0.2661 as of posted documentation today, 5.0.0.3001 is the latest tag today).

[prjemian]

Update the conda-forge feedstock. The feedstock recipe pulls its version from https://github.com/picca/hkl but the latest version there is the same 2173. Revise the feedstock to pull from the contemporary repo: https://repo.or.cz/hkl.git

[prjemian]

pattern:

name: hkl
version: 5.0.0.3001
sha256: 0be5985a8aa893ab9aaffbfb8dc08fce0e4a2d79

---: https://github.com/picca/{{ name }}/archive/v{{ version }}.tar.gz
+++: https://repo.or.cz/{{ name }}.git/snapshot/{{ sha256 }}.tar.gz

[prjemian]

Might be simple (simpler than imagined) to update the conda-forge recipe but I should test things first locally with the new version before updating the feedstock.

[prjemian]

Making some progress constructing a Docker image:

FROM ubuntu
LABEL Description="Build libhkl"

USER root
WORKDIR /root
ENV HOME /root

SHELL ["/bin/bash", "-c"]

RUN apt-get update && apt-get -y --no-install-recommends install \
    apt-utils \
    autoconf \
    automake \
    bison \
    build-essential \
    clang \
    cmake \
    flex \
    gdb \
    gsl-bin \
    gtk-doc-tools \
    less \
    libcairo2-dev \
    libffi-dev \
    libglib2.0-dev \
    libgsl0-dbg \
    libgsl0-dev \
    meson \
    nano \
    pkg-config \
    wget

# name: hkl
# version: 5.0.0.3001
# sha256: 0be5985a8aa893ab9aaffbfb8dc08fce0e4a2d79
# https://repo.or.cz/{{ name }}.git/snapshot/{{ sha256 }}.tar.gz
COPY hkl-5.0.0.3001.tar.gz .

RUN tar xzf hkl-5.0.0.3001.tar.gz
WORKDIR hkl-0be5985/
RUN uname -a
RUN ls -lAFgh .

Next step is to build a configure file and that's when the trouble starts:

docker build -t hkl_image .
# ....
docker run --rm -it hkl_image
root@59d2dfea95aa:~/hkl-0be5985# ./autogen.sh
autoreconf: export WARNINGS=
autoreconf: Entering directory '.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
configure.ac:214: error: AM_COND_IF: no such condition "HAVE_INTROSPECTION"
/usr/share/aclocal-1.16/cond-if.m4:23: AM_COND_IF is expanded from...
configure.ac:214: the top level
autom4te: error: /usr/bin/m4 failed with exit status: 1
aclocal: error: /usr/bin/autom4te failed with exit status: 1
autoreconf: error: aclocal failed with exit status: 1
root@59d2dfea95aa:~/hkl-0be5985# 

[prjemian]

Not sure how to resolve this error and move forward:

configure.ac:214: error: AM_COND_IF: no such condition "HAVE_INTROSPECTION"

@picca, @mrakitin, @klauer : Any hints?

[klauer]

Don't you need everything from hkl as well? https://github.com/conda-forge/hkl-feedstock/blob/main/recipe/meta.yaml#L31 (Isn't that message related to gobject-introspection?)

[prjemian]

Got confused about some of those packages (glib was not found). This solved the one problem and I'll keep trying the other packages to get a build going. Thanks!

[prjemian]

Getting further, now:

./configure: line 19635: syntax error near unexpected token `-fanalyzer,'
./configure: line 19635: `  AX_CHECK_COMPILE_FLAG(-fanalyzer, CFLAGS="$CFLAGS -fanalyzer", , )'

[prjemian]

libgtk-3-dev - not the solution to the fanalyzer problem, but needed.

With these lines commented (or removed) from configure.ac, then ./autogen.sh && ./configure completes without errors.

OPTION_DEFAULT_OFF([analyzer], [enable gcc-10 -fanalyzer])
AM_CONDITIONAL([ANALYZER], [test x$enable_analyzer != xno])
AM_COND_IF([ANALYZER],
        AX_CHECK_COMPILE_FLAG([-fanalyzer], [CFLAGS="$CFLAGS -fanalyzer"], [], [])
)

[prjemian]

Static analyzer: https://gcc.gnu.org/onlinedocs/gcc/Static-Analyzer-Options.html

[klauer]

Perhaps related: https://stackoverflow.com/questions/30412576/autotools-syntax-error-with-ax-check-compile-flag

[prjemian]

I'll try that, thanks

On Thu, Mar 16, 2023, 11:05 AM Ken Lauer @.***> wrote:

Perhaps related: https://stackoverflow.com/questions/30412576/autotools-syntax-error-with-ax-check-compile-flag

— Reply to this email directly, view it on GitHub https://github.com/bluesky/hklpy/issues/250#issuecomment-1472263359, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARMUMAOS57CYHPTAQQP6L3W4M26FANCNFSM6AAAAAAV4M4PAU . You are receiving this because you authored the thread.Message ID: @.***>

[prjemian]

Yup, that removes the problem with fanalyzer

[prjemian]

Now I have a dockerized build process that completes with no errors:

# https://github.com/bluesky/hklpy/issues/250

FROM ubuntu
LABEL Description="Build libhkl"

USER root
WORKDIR /root
ENV HOME /root

# name: hkl
# version: 5.0.0.3001
# sha256: 0be5985a8aa893ab9aaffbfb8dc08fce0e4a2d79

# ---: https://github.com/picca/{{ name }}/archive/v{{ version }}.tar.gz
# +++: https://repo.or.cz/{{ name }}.git/snapshot/{{ sha256 }}.tar.gz
ENV VERSION 5.0.0.3001
ENV SHA 0be5985a8aa893ab9aaffbfb8dc08fce0e4a2d79
ENV TARNAME hkl-0be5985
ENV TARBALL "hkl-${VERSION}.tar.gz"
ENV URL "https://repo.or.cz/hkl.git/snapshot/${SHA}.tar.gz"

# docker build -t hkl_image . && docker run --rm -it hkl_image

SHELL ["/bin/bash", "-c"]

RUN apt-get update && apt-get -y --no-install-recommends install \
    apt-utils \
    autoconf \
    autoconf-archive \
    automake \
    build-essential \
    clang \
    cmake \
    flex \
    gdb \
    gobject-introspection \
    gsl-bin \
    gtk-doc-tools \
    less \
    libglib2.0-dev \
    libgsl0-dbg \
    libgsl0-dev \
    libgtk-3-dev \
    libhdf5-dev \
    libtool \
    nano \
    pkg-config \
    wget

# normally, download, for now COPY from local to avoid repeated downloads
# RUN wget https://repo.or.cz/hkl.git/snapshot/${SHA}.tar.gz -o "${TARBALL}"
COPY "${TARBALL}" .

RUN tar xzf "${TARBALL}"
RUN /bin/rm "${TARBALL}"
WORKDIR "${TARNAME}/"
RUN uname -a
RUN ls -lAFgh .

RUN ./autogen.sh
RUN ./configure --disable-hkl-doc --disable-gtk-doc
RUN make

[prjemian]

Next, setup a conda environment with hklpy (in the image) and use the new material (library and header files).

[prjemian]

Basic python check that hklpy can be imported and show something meaningful:

from hkl.geometries import SimulatedE4CV

fourc = SimulatedE4CV("", name="fourc")
fourc.wh()

Dockerfile additions which set up the conda environment:

# normally, download, for now COPY from local to avoid repeated downloads
# RUN wget --no-check-certificate https://repo.anaconda.com/Miniconda3-py310_23.1.0-1-Linux-x86_64.sh -o /tmp/Miniconda.sh
# RUN wget --no-check-certificate https://repo.anaconda.com/Miniconda3-latest-Linux-x86_64.sh -o /tmp/Miniconda.sh
COPY Miniconda3-latest-Linux-x86_64.sh /tmp/Miniconda.sh
RUN bash /tmp/Miniconda.sh -b -p /root/miniconda
RUN source /root/miniconda/bin/activate \
    && conda update -y conda \
    && conda create -n hklpy -c conda-forge hklpy

COPY hklpy_check.py /root/
RUN source /root/miniconda/bin/activate hklpy && python /root/hklpy_check.py

[prjemian]

So far, so good (unmodified libraries):

Step 26/26 : RUN source /root/miniconda/bin/activate hklpy && python /root/hklpy_check.py
 ---> Running in 88a2d8d94b14
===================== ========= =========
term                  value     axis_type
===================== ========= =========
diffractometer        fourc              
sample name           main               
energy (keV)          8.05092            
wavelength (angstrom) 1.54000            
calc engine           hkl                
mode                  bissector          
h                     0.0       pseudo   
k                     0.0       pseudo   
l                     0.0       pseudo   
omega                 0         real     
chi                   0         real     
phi                   0         real     
tth                   0         real     
===================== ========= =========

[prjemian]

@klauer Do you have suggestions how to replace the libhkl library and other necessary files? Update with these files, then retest?

root@12262cf8a801:~/hkl-0be5985# ll ./hkl/.libs/libhkl.so*
lrwxrwxrwx 1 root root      15 Mar 16 22:04 ./hkl/.libs/libhkl.so -> libhkl.so.5.0.0*
lrwxrwxrwx 1 root root      15 Mar 16 22:04 ./hkl/.libs/libhkl.so.5 -> libhkl.so.5.0.0*
-rwxr-xr-x 1 root root 1440256 Mar 16 22:04 ./hkl/.libs/libhkl.so.5.0.0*
root@12262cf8a801:~/hkl-0be5985# ll hkl.h
-rw-rw-r-- 1 root root 23995 May 23  2022 hkl.h
root@12262cf8a801:~/hkl-0be5985# ll hkl/ccan/*.h
-rw-r--r-- 1 root root 2433 Mar 16 22:04 hkl/ccan/ccan_config.h
root@12262cf8a801:~/hkl-0be5985# ll hkl/ccan/darray/*.h
-rw-rw-r-- 1 root root 12531 May 23  2022 hkl/ccan/darray/darray.h

[prjemian]

Will build conda package (per the recipe in meta.yaml) with new library, et al. and install with conda, all in the Dockerfile.

[klauer]

That seems a reasonable approach, @prjemian

[prjemian]

• https://github.com/conda-forge/hkl-feedstock/pull/13

[prjemian]

Working on a solution now. Build the libhkl.tar.gz library without conda. Then, untar this file into $CONDA_PREFIX and build the conda recipe. Choose matching versions of gobject-introspection and gsl for both the build of libhkl and the build of the conda-forge recipe.

[prjemian]

Also, found a way to get the libhkl version string into the libhkl.tar.gz, by modifying the hkl.h file with the version tag (inserts the version definition line before the line with text tau = 2pi):

# add version info
sed -i '/^.*tau = 2pi.*/i #define HKL_VERSION "'"${HKL_TAG}"'"' hkl.h

This makes it all the way into hklpy: hkl.util.libhkl.VERSION='v5.0.0.3001'

[prjemian]

Apparently, the HKL_VERSION definition in hkl.h is accessed by the gobject-introspection package, which strips off the HKL_ part before delivering that through the Python API.

[prjemian]

This procedure was revealed by examining dir(hkl.util.libhkl) in the Python code. The HKL_TAU definition appears in Python as hkl.util.libhkl.TAU. Nice to see that HKL_VERSION appears in Python similarly, as hkl.util.libhkl.VERSION

[mrakitin]

Nice progress, @prjemian!

[prjemian]

Done.:

(hklpy) prjemian@arf:~/.../Bluesky/hklpy$ conda search hkl | grep 3001                                                                        
hkl                       5.0.0.3001 py310hff52083_0  conda-forge         
hkl                       5.0.0.3001 py311h38be061_0  conda-forge         
hkl                       5.0.0.3001  py38h578d9bd_0  conda-forge         
hkl                       5.0.0.3001  py39hf3d152e_0  conda-forge