search

mpimd-csc / flexiblas

FlexiBLAS - A BLAS and LAPACK wrapper library with runtime exchangeable backends. This is only a mirror of https://gitlab.mpi-magdeburg.mpg.de/software/flexiblas-release
https://www.mpi-magdeburg.mpg.de/projects/flexiblas
GNU Lesser General Public License v3.0
41 stars 7 forks source link
blas lapack linear-algebra linear-algebra-library

readme

FlexiBLAS - A BLAS and LAPACK wrapper library with runtime exchangeable backends

Version 3.4.4 - DOI: 10.5281/zenodo.11260086

Project Website: https://www.mpi-magdeburg.mpg.de/projects/flexiblas

Git Repository: https://gitlab.mpi-magdeburg.mpg.de/software/flexiblas-release

Copyright 2013-2024 by Martin Köhler (0000-0003-2338-9904) and Jens Saak (0000-0001-5567-9637)

Documentation

This Documentation includes:

About Flexiblas

FlexiBLAS is a wrapper library that enables the exchange of the BLAS (Basic Linear Algebra System) and LAPACK (Linear Algebra PACKage) implementation used in an executable without recompiling or re-linking it. This could in principle also be achieved using the LD_LIBRARY_PATH mechanism, if all BLAS and LAPACK were consistently implemented as a single library file with a fully compatible interface. Unfortunately, not all BLAS and LAPACK libraries are realized as only this exact one shared library containing all required symbols. Implementations with differing numbers of library files (shared objects) clearly break this mechanism of easily switching via the LD_LIBRARY_PATH environment variable.

The FlexiBLAS library provides a GNU Fortran compatible interface to all functions and subroutines provided by the Netlib reference implementations. As backends FlexiBLAS can employ all BLAS and LAPACK implementations which consist of a single shared library directly. Other variants like the Intel MKL or ATLAS that use multiple files are integrated by FlexiBLAS by wrapping all files into a single surrogate library.

Install

The build system used by FlexiBLAS is cmake which prefers an out-of-source build:

    mkdir build
    cd build
    cmake ../ <FURTHEROPTIONS>
    make
    make install

If the build system should try to build backends for BLAS and LAPACK libraries that are not known to the FindBLAS cmake module, or reside in non-standard paths on your system, make sure to include their location (containing directory) to the LD_LIBRARY_PATH environment variable. At the end of the configuration step, cmake will inform you about the additional backends found on your system.

The configuration via cmake can be influenced further by the following options:

The PROFILE=ON/OFF option was removed from version 1.1.0 onward. Beginning with version 3.0.0 profiling is done using a hook functionality and is no longer compiled in FlexiBLAS directly. See the Profiling section for details.

The following BLAS backends are tested during development:

The following compilers are tested:

Other compilers might work, but not tested.

Setup with precompiled reference BLAS and LAPACK

On many Linux and BSD operating systems static versions of BLAS and LAPACK can be installed from the package management system. These versions can be used instead of FlexiBLAS's internal fallback implementation. In order to use a provided static library providing a BLAS API it needs to be ensured that this library is compiled with support for position-independent code (-fPIC).

The BLAS implementation is set in the configuration procedure via

    cmake -DSYS_BLAS_LIBRARY=/absolute/path/to/liblas.a

The BLAS library has to provide all functions and subroutines available in the reference BLAS implementation since LAPACK 3.0.

The LAPACK fallback implementation can be provided in the same way via

    cmake -DSYS_LAPACK_LIBRARY=/absolute/path/to/liblapack.a \
          -DLAPACK_API_VERSION="LAPACK-VERSION"

The LAPACK library must provide all symbols that the reference implementation from NETLIB with the given version provides. A list of supported LAPACK versions are given above. If the version is not specified cmake tries to obtain it with a call to LAPACK's ILAVER. Note that the path must be absolute, the ~-operator to reference the home directory is not allowed.

Setup with Custom BLAS and LAPACK Implementations.

By default, FlexiBLAS tries to locate as many BLAS and LAPACK installations as possible on your system. If you want to add our own ones to the default setup you can pass the -DEXTRA option to cmake. It specifies a semicolon-separated list of additional BLAS names that should be added. For each additional library name name in the list, a path needs to be specified via -Dname_LIBRARY=PATH_TO_NAME. If special linker flags are required the option -Dname_LINKER_FLAGS="FLAGS" can be passed to cmake. For example, if a custom build OpenBLAS-0.3.4 and OpenBLAS-0.3.10 should be included, the following options have to be added to cmake:

    -DEXTRA="OpenBLAS034;OpenBLAS0310" \
    -DOpenBLAS034_LIBRARY="/home/user/openblas-0.3.4/libopenblas.a;gomp;pthread;" \
    -DOpenBLAS0310_LIBRARY="/home/user/openblas-0.3.10/libopenblas.a;gomp;pthread;"

Setup on Debian and Derivatives

The easiest way is to build a Debian package out of the source and install it. At the moment we support the package creation on Ubuntu 20.04 and Debian 11 (Testing) since they provide OpenBLAS and BLIS split in different packages depending on the used threading model. In order to build the package the following prerequisites need to be installed:

    sudo apt-get install debhelper cmake gfortran gcc libatlas-base-dev \
    libopenblas-openmp-dev libopenblas-pthread-dev libopenblas-serial-dev \
    libopenblas64-openmp-dev libopenblas64-pthread-dev \
    libopenblas64-serial-dev libblis-openmp-dev libblis-pthread-dev \
    libblis-serial-dev libblis64-openmp-dev libblis64-pthread-dev \
    libblis64-serial-dev libblas-dev libblas64-dev liblapack-dev \
    liblapack64-dev

Afterwards, you can build the Debian package out of the source code by typing:

    fakeroot dpkg-buildpackage -us -uc

inside the extracted source. After building the package the source directory can be cleaned using

    debian/rules clean

The build process creates a set of packages that are available one directory level above. They are installed using:

    sudo dpkg -i ../libflexiblas*.deb

If you update from FlexiBLAS <= 2.0.0 please remove all related packages and configuration files before.

Setup on Fedora

If you are using Fedora Linux, you can install FlexiBLAS easily using dnf since FlexiBLAS is part of Fedora Linux. Just type

dnf install flexiblas

This installs most likely one of the latest FlexiBLAS releases.

Thanks Iñaki Ucar (https://src.fedoraproject.org/user/iucar) for packaging and maintaining the Fedora integration.

Caveats

Testing

FlexiBLAS comes with the reference BLAS and reference LAPACK test suite. The tests are executed using

     make test

and use the NETLIB BLAS implementation by default in order to check compliance with the reference implementation. Other BLAS and LAPACK implementations can be tested by setting the FLEXIBLAS_TEST environment variable. The variable has the same meaning as the FLEXIBLAS environment variable but only for the tests. If single tests need to be selected we refer to the help of the cmake ctest utility.

Configuration Files

FlexiBLAS reads a set of configuration files at startup. These are:

If FlexiBLAS is compiled with the INTEGER8 option enabled, flexiblasrc is replaced by flexiblasrc64.

    CMAKE_INSTALL_PREFIX/etc/flexiblasrc

and:

    ~/.flexiblasrc

or:

    CMAKE_INSTALL_PREFIX/etc/flexiblasrc64

and:

    ~/.flexiblasrc64

in case of a 64-bit integer build.

The files contain a mapping between the name of a backend and the corresponding name of the shared object. The setting in the user-owned file overrides the global settings. All settings are superseded eventually by setting environment variables. The syntax of the file is similar to the KDE configuration files or the ini-files known from early MS-Windows versions. Basically, it is a key-value store with section support. In the first section of the configuration file are global definitions like the default BLAS backend or the verbosity level:

    default = NETLIB
    verbose = 0

Then for each BLAS backend, there is a block of the following style:

    [NAME_OF_THE_BACKEND]
    library = shared_object.so
    comment = Information text about the BLAS backend

If the library argument defines a relative path, FlexiBLAS searches inside its default path (CMAKE_INSTALL_PREFIX/lib/flexiblas or CMAKE_INSTALL_PREFIX/lib/flexiblas64) for this shared object. If the path is absolute it tries to open the library directly. The name of the backend is case-insensitive!

Additionally, one can define additional search paths in the first section. Therefore one has to add a key-value entry of the following style to the first section:

    pathXXX = /additional/search/path

where the XXX part of the key should be an increasing unique integer for each additional search path.

If no default mapping is set in one of the configuration files, FlexiBLAS will use the standard Netlib implementation.

The user config files can be handled using the flexiblas tool. This tool can manage the user and the system configuration file, excluding adding additional search paths.

The flexiblas tool can be used for example to:

If the name of the backend is not given, the default setting is removed.

New BLAS backends can also be added to the configuration files using the flexiblas tool. For this please look at the output of

    flexiblas help

Caution: In case of a 64-bit integer build the flexiblas tool is renamed to flexiblas64 in order to setup both, a 32-bit and a 64-bit version in parallel.

Selecting the Backend at Runtime

The behavior of the FlexiBLAS wrapper is controlled by a set of environment variables described below and in the above-mentioned configuration files. The most important environment variable is called FLEXIBLAS. This variable describes the shared object library file where the backend BLAS implementation is contained. The variable supports three types of descriptions. If the value does not contain a / FlexiBLAS searches for the given name in CMAKE_INSTALL_PREFIX/lib/flexiblas. That means the value is interpreted as the name of one of the wrapper libraries. If it does not find it there, it searches the configuration files for a line matching the value and fetches the actual name of the shared object library from there. Otherwise, the value is interpreted as a full path description of the BLAS backend library shared object file to use. If the variable is not set, the Netlib reference BLAS, or the default from the configuration files is used. See the documentation for details on the names, syntax, and locations of the configuration files.

Usage Example

If FlexiBLAS is built with an interface to the ATLAS library then one can use the following in an application to call the ATLAS instead of the default BLAS:

    FLEXIBLAS="libblas_atlas.so" ./yourapp

In case FlexiBLAS should use OpenBLAS and is registered as OpenBLAS in one of the configuration files the application can be started as follows:

    FLEXIBLAS=OpenBLAS ./yourapp

Remember you can list all installed backends with:

    flexiblas list

Linking a Program Against FlexiBLAS

By default FlexiBLAS produces a library called libflexiblas.so, which one can link against a program just like the reference BLAS implementation. Other libraries are not necessary.

Environment Variables

Attention: In the case of a 64-bit integer build, all environment variables begin with FLEXIBLAS64 instead of FLEXIBLAS.

64-Bit Integer Build

As already mentioned in the case when the 64-bit integer interface is enabled, the names of all libraries, directories, and environment variables are suffixed by 64, that means in order to link against the 64-bit library, you have to use:

    gcc -lflexiblas64 ...

The flexiblas tool is now named flexiblas64. This allows to installed of both variants on a single system.

Profiling

Since version 3.0.0 the profiling framework is no longer compiled into the library. It is now realized using the function overloading support of FlexiBLAS. The profiling is enabled by loading libflexiblas_hook_profile.so in the for function overloading. This is done using the FLEXIBLAS_HOOK environment variable before starting a problem.

The profiling output can be controlled using the following environment variables:

Octave Interface

The GNU Octave interface is a separate package that can be downloaded from:

http://www.mpi-magdeburg.mpg.de/projects/flexiblas

and installed via:

    pkg install -verbose flexiblas-octave.tar.gz

from the command-line prompt in GNU Octave. After installing the package you can get an overview of the installed functions by calling:

    pkg describe -verbose flexiblas

from the GNU Octave command-line prompt. This package only works on Linux and BSD versions of GNU Octave, but NOT on any version of MS-Windows!

Backend Builder

In order to build backends later without reconfiguring or rebuilding FlexiBLAS, a CMake helper tool exists in tools/backend_builder. This tool can be used to build your own backends individually.

Cite As

M. Köhler, J. Saak. FlexiBLAS - A flexible BLAS library with runtime exchangeable backends. LAPACK Working Note: 284, 2013. https://www.netlib.org/lapack/lawnspdf/lawn284.pdf

Acknowledgments

We thank Christian Himpe (0000-0003-2194-6754) for documenting, testing, and providing feedback.

We also thank Iñaki Ucar (0000-0001-6403-5550) for creating a Fedora package and pointing out Linux software maintainer-related issues with the configuration and building process.

Bug Reports

If you find a bug in FlexiBLAS please prepare a minimal examples which reproduces the bug. Without minimal examples, we cannot fix the bug. Send feedback to:

koehlerm@mpi-magdeburg.mpg.de

License

The whole library is provided under the conditions of the LGPLv3 and later The testing code ( contained in test/ ) and the Reference Implementation implementation (contained in contributed/) are covered by the license of LAPACK ( https://www.netlib.org/lapack ). See COPYING.NETLIB for more details about this license.