Resolving 32/64 and 64/64 libraries without function suffix

[BrandonGroth]

I am attempting to port our BLAS/LAPACK shared libraries to work with the lbt_forward command. Our library is split into 4 shared libraries where we built them with (serial or SMP) and (32/64 or 64/64) options. However, our 64/64 libraries do not have any additional naming convention; both the 32/64 and 64/64 libraries use the same function names ( ie dgemm() ).

From my understanding, any pre-packaged Julia distribution forces users to use 64/64 BLAS. Since our 64/64 libraries do not have a suffix in any function, the lbt_forward command seems to call the 32/64 variant w/ 64-bit integers and causes a segfault at setup. In the ReadMe ABI section, it sounds like LBT require BLAS libraries to use the 64/64 OpenBLAS naming convention via the suffix. While possible on our end, is there any other way to resolve this?

Current version: 1.7.0-rc2 (last version prebuilt for powerpcle). Possible to move to 1.7.2 or higher if needed.

[jd-foster]

@BrandonGroth : could you share any details on whether you resolved this, and if so, how you did so?

[BrandonGroth]

@jd-foster

Our original shared object files had the BLAS symbol names <func> or <func>_ in both LP64 and ILP64 modes. The LP64 shared object library never had an issue w/ lbt_forward, but our ILP64 library would not be accepted by lbtforward. After reading the documentation and doing objdump on the libopenblas64.so, we realized we had to rename all exported routines to <func>_64_ in the ILP64 shared object file. To do this, we used the objcopy utility w/ the redefine-sym option. For example:

objcopy --redefine-sym <func>_=<func>_64_ <current .o for func>  <new .o for func>

We are only doing this for Julia because our "current" shared object builds seems to work with other frameworks like Python and R without issue.

We also had to add exports for all the internal LAPACK routines like isamaxsub. We didn't have these exported because they were used internally, but LBT required them be available.

Finally, we used the lbt_forward that was similar to MKL.jl:

BLAS.lbt_forward(<path to ILP64 .so>; clear=true, verbose=true, suffix_hint="64_")
BLAS.lbt_forward(<path to LP64 .so>; verbose=true, suffix_hint="")

Originally, it was very confusing to understand what suffix was required for the symbols. The README says _64, but then libopenblas uses 64_. It does seem that the suffix detector can find both, but mixing signals makes it unclear what is the right course of action.

Unfortunately, there still isn't any way to use LP64 mode on our system. The USE_BLAS64=0 option can fail to build any system outside of the ones that have special code in the Makefile. I have detailed it here, but hasn't seen any movement or interest: https://github.com/JuliaLang/julia/issues/47849

[jd-foster]

Thank you for that detailed explanation, it's good to know that I wasn't alone in struggling with these aspects.