pyhdk import fails on the HDK 0.9 from conda

[Egor-Krivov]

I have a problem with the latest conda release of HDK.

Source code to reproduce:

import pyhdk

pyhdk.init()

Error msg:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007feb5ac04210, pid=503413, tid=503526
#
# JRE version: OpenJDK Runtime Environment (11.0.1+13) (build 11.0.1+13-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.1+13-LTS, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# C  [libjimage.so+0x4210]  ImageStrings::find(Endian*, char const*, int*, unsigned int)+0x70
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to /localdisk/ekrivov/db-benchmark/modin/core.503413)
#
# An error report file with more information is saved as:
# /localdisk/ekrivov/db-benchmark/modin/hs_err_pid503413.log
#
# If you would like to submit a bug report, please visit:
#   http://www.azulsystems.com/support/
#
Aborted (core dumped)

Code to reproduce after creating hdk_bug.py with source code from above:

#!/bin/bash
set -e

curl -o install_miniconda.sh -L https://repo.anaconda.com/miniconda/Miniconda3-py311_23.9.0-0-Linux-x86_64.sh && \
    sh install_miniconda.sh -u -b -p ./miniconda && \
    rm -f install_miniconda.sh

source ./miniconda/bin/activate
conda install -y conda-libmamba-solver

# install binaries
conda install -y -c conda-forge modin-hdk --solver=libmamba

python hdk_bug.py

[Egor-Krivov]

BTW, hdk works, we utilize it with modin. It does execute queries, bug happens when the script exits

[Egor-Krivov]

Assigned to Alex on advice from Ilya

[Garra1980]

Probably this is the reason - https://github.com/conda-forge/hdk-feedstock/commit/7a402c0a1b01710a65ac4a40ea31222e1bedb471

[alexbaden]

I can confirm updating to openjdk 17 fixes this issue. So, we need to modify conda-forge to pin openjdk >= 17 instead of 11. I will open the conda-forge PR and report back.

[alexbaden]

https://github.com/conda-forge/hdk-feedstock/pull/54 @Garra1980 @leshikus please take a look.