flame / blis

BLAS-like Library Instantiation Software Framework
Other
2.18k stars 362 forks source link

Not possible to link Blis and Lapack statically into the same executable #816

Open tbetcke opened 1 week ago

tbetcke commented 1 week ago

I would like to link Blis and Lapack statically into the same executable. However, this is not possible since several symbols are multiply defined between Blis and Lapack. These are lsame, xerbla, xerbla_array, crot, czymv, csyr, zrot, zsymv, zsyr.

I noticed an earlier bug report #776 in which some of these were discussed and then included into Blis. Would it be possible to have an option to disable this in the future?

The background is that I am integrating them into a Rust project. Rust prefers static linkage and does not handle rpath well for shared libraries.

fgvanzee commented 5 days ago

We also just handled a similar request via #812. Your request, @tbetcke, has me wondering about a general-purpose solution that combines the compatibility accommodations for both 812 and your request into a single option, say, --disable-symbols=<comma-separated-list-of-symbols>. Then the user could omit whichever symbols they wish. (Of course, we'd have to guard the eligible symbols with cpp macros, but that's not difficult and it's something we can do incrementally as these offending symbols are brought to our attention.)

What do you think, @devinamatthews?

devinamatthews commented 5 days ago

Yes, that's an interesting idea but it would be a lot of work to make it work "seamlessly" for end-users (e.g. they can disable any symbols within certain classes), and would be surprising to users if the list of symbols which can be disabled is small, irregularly defined, and/or constantly changing.

nick-knight commented 5 days ago

As an author of applications that use BLIS, LAPACK, and ScaLAPACK together, I'm generally in favor of improving interoperability. Beyond resolving multiply defined symbols, what I'd really like is for BLIS implementations to override those in Sca/LAPACK. A partial solution would be for BLIS to name-mangle select routines with __wrap_, so I can use --wrap to prefer BLIS implementations. I believe that this solution is only partial because I don't see how it could hijack the libcalls inside the Sca/LAPACK binaries; achieving that I think would require build-system modifications in both BLIS and Sca/LAPACK. I don't know the best way forward to request such cross-project functionality.

rvdg commented 5 days ago

I think the point Nick is making: the implementation in a BLAS library is likely to be higher performing than the reference implementation in LAPACK or ScaLAPACK (or some other library). Hence, it is such a library that should selectively remove rather than, say, BLIS.

On the other hand, XERBLAS may be a replacement for the one in the BLAS specifically to do more meaningful error handling. In that case, it is the one in LAPACK/ScaLAPACK that should overwrite.

This is, I guess, an argument for vertical integration of such libraries... A project we intend to pursue next.

devinamatthews commented 5 days ago

For the computational routines, one can access the BLIS version using the BLIS object or typed APIs. For stuff like lsame and xerbla there is no really good solution (esp. for xerbla since that is supposed to be overridden by user applications.

leekillough commented 5 days ago

Is it possible to mitigate the problem with __attribute__((weak))?

devinamatthews commented 5 days ago

@tbetcke does linking with -Wl,-z,muldefs work?