mpimd-csc / flexiblas

FlexiBLAS - A BLAS and LAPACK wrapper library with runtime exchangeable backends. This is only a mirror of https://gitlab.mpi-magdeburg.mpg.de/software/flexiblas-release
https://www.mpi-magdeburg.mpg.de/projects/flexiblas
GNU Lesser General Public License v3.0
36 stars 7 forks source link

3.1.0 build failures #20

Closed Enchufa2 closed 2 years ago

Enchufa2 commented 2 years ago

I sent scratch builds to Fedora 35 and rawhide:

$ ll /usr/lib64/libflexiblas*
lrwxrwxrwx. 1 root root      23 jul 22 02:19 /usr/lib64/libflexiblas64_api.so -> libflexiblas64_api.so.3
lrwxrwxrwx. 1 root root      25 jul 22 02:19 /usr/lib64/libflexiblas64_api.so.3 -> libflexiblas64_api.so.3.0
-rwxr-xr-x. 1 root root   15960 jul 22 02:20 /usr/lib64/libflexiblas64_api.so.3.0
lrwxrwxrwx. 1 root root      24 jul 22 02:19 /usr/lib64/libflexiblas64_mgmt.so -> libflexiblas64_mgmt.so.3
lrwxrwxrwx. 1 root root      26 jul 22 02:19 /usr/lib64/libflexiblas64_mgmt.so.3 -> libflexiblas64_mgmt.so.3.0
-rwxr-xr-x. 1 root root   49760 jul 22 02:20 /usr/lib64/libflexiblas64_mgmt.so.3.0
lrwxrwxrwx. 1 root root      19 jul 22 02:19 /usr/lib64/libflexiblas64.so -> libflexiblas64.so.3
lrwxrwxrwx. 1 root root      21 jul 22 02:19 /usr/lib64/libflexiblas64.so.3 -> libflexiblas64.so.3.0
-rwxr-xr-x. 1 root root 3880056 jul 22 02:20 /usr/lib64/libflexiblas64.so.3.0
lrwxrwxrwx. 1 root root      21 jul 22 02:19 /usr/lib64/libflexiblas_api.so -> libflexiblas_api.so.3
lrwxrwxrwx. 1 root root      23 jul 22 02:19 /usr/lib64/libflexiblas_api.so.3 -> libflexiblas_api.so.3.0
-rwxr-xr-x. 1 root root   15960 jul 22 02:20 /usr/lib64/libflexiblas_api.so.3.0
lrwxrwxrwx. 1 root root      22 jul 22 02:19 /usr/lib64/libflexiblas_mgmt.so -> libflexiblas_mgmt.so.3
lrwxrwxrwx. 1 root root      24 jul 22 02:19 /usr/lib64/libflexiblas_mgmt.so.3 -> libflexiblas_mgmt.so.3.0
-rwxr-xr-x. 1 root root   49752 jul 22 02:20 /usr/lib64/libflexiblas_mgmt.so.3.0
lrwxrwxrwx. 1 root root      17 jul 22 02:19 /usr/lib64/libflexiblas.so -> libflexiblas.so.3
lrwxrwxrwx. 1 root root      19 jul 22 02:19 /usr/lib64/libflexiblas.so.3 -> libflexiblas.so.3.0
-rwxr-xr-x. 1 root root 3872616 jul 22 02:20 /usr/lib64/libflexiblas.so.3.0

and now:

Processing files: flexiblas-netlib-3.1.0-1.fc35.x86_64
error: File not found: /builddir/build/BUILDROOT/flexiblas-3.1.0-1.fc35.x86_64/usr/lib64/libflexiblas.so.3.1
error: File not found: /builddir/build/BUILDROOT/flexiblas-3.1.0-1.fc35.x86_64/usr/lib64/libflexiblas_api.so.3.1
error: File not found: /builddir/build/BUILDROOT/flexiblas-3.1.0-1.fc35.x86_64/usr/lib64/libflexiblas_mgmt.so.3.1

Is this intentional? Should I just remove the library files with the minor version?

/usr/bin/ld: /tmp/cccPyAll.ltrans0.ltrans.o: in function `main':
    /usr/share/cmake/Modules/FortranCInterface/Verify/main.c:14: undefined reference to `VerifyFortran'
    collect2: error: ld returned 1 exit status

See the complete log e.g. here. Relevant differences? In F35 we have gcc 11.2.1; in rawhide, there's gcc 12.0.0.

Enchufa2 commented 2 years ago

BTW, there's a thread in the devel list about gcc 12. If this could be a bug in gcc, let me know to report it there.

grisuthedragon commented 2 years ago

For the first one it is correct that the version of the libraries are increased since there a some new symbols in. That allows old programs to use the new libraries without relinking but code that uses the new ones should not use the 3.0 library version.

For the second one I took a look into the build logs and it seems, as in the thread you linked, that this is a problem of gcc 12 and cmake. Is there a way to use FC rawide with gcc 12 in docker for checking it?

Enchufa2 commented 2 years ago

For the first one it is correct that the version of the libraries are increased since there a some new symbols in. That allows old programs to use the new libraries without relinking but code that uses the new ones should not use the 3.0 library version.

I mean, the issue is that RPM expects files with 3.1 appended, as listed in the SPEC, but apparently these are not generated in the new version (thus the error: file not found).

For the second one I took a look into the build logs and it seems, as in the thread you linked, that this is a problem of gcc 12 and cmake. Is there a way to use FC rawide with gcc 12 in docker for checking it?

There's no rawhide compose with gcc 12 yet, but you can manually download the latest build from here and install it in a rawhide container.

Enchufa2 commented 2 years ago

FWIW, Koschei reports that version 3.0.4 fails with the same error with gcc 12: https://koschei.fedoraproject.org/package/flexiblas?collection=f36

grisuthedragon commented 2 years ago

The first one is fixed in 3.1.1 (uploaded within the next minutes) I forgot to update the so-version in the CMakeFiles.

The second one is specific to CMake + GCC/GFortran 12 + CMake's FortranC Interface. I created the following MWE, which produces the crash without requiring anything from FlexiBLAS.

cat << EOL > CMakeLists.txt
PROJECT(F-C-MWE LANGUAGES C Fortran) 
include(FortranCInterface)
FortranCInterface_HEADER(FCMangle.h
                         MACRO_NAMESPACE "FC_"
                         SYMBOL_NAMESPACE "FC_"
                         SYMBOLS mysub mymod:my_sub)
FortranCInterface_VERIFY()
EOL

mkdir -p build
cd build 
# Set the Flags from the FC build server
CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection'
export CFLAGS
CXXFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection'
export CXXFLAGS
FFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS  -fstack-protector-strong  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -I/usr/lib64/gfortran/modules'
export FFLAGS
FCFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -I/usr/lib64/gfortran/modules'
export FCFLAGS
LDFLAGS='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -Wl,--build-id=sha1'
export LDFLAGS

# configure
cmake ../ 

When you verify that the first part is closed. I close this bug since the second has nothing to do with FlexiBLAS.

Enchufa2 commented 2 years ago

What is flexiblas-config? Is it the flexiblas utility but renamed?

Enchufa2 commented 2 years ago

Oh, and for 64-bit systems, in the 64-bit build,

error: Installed (but unpackaged) file(s) found:
   /usr/bin/-config
   /usr/bin/flexiblas-config

-config is generated instead of flexiblas64-config. Or maybe this is an error in my SPEC? Complete log: https://kojipkgs.fedoraproject.org//work/tasks/2955/81362955/build.log

grisuthedragon commented 2 years ago

The flexiblas-config tool is sometihng similar to the octave-config etc. It gives output for easy building application with FlexiBLAS like

 gcc `flexiblas-config --cflags` ...

It should be part of the flexiblasXX-devel package. The missing flexiblas64 is indeed an error. Fixed within the evening.

Enchufa2 commented 2 years ago

Looking good now, thanks! I'll update F34 and F35 to v3.1.2 and wait until the gcc thing is resolved for rawhide. Thanks for the MWE above too!

Enchufa2 commented 2 years ago

The second one is specific to CMake + GCC/GFortran 12 + CMake's FortranC Interface. I created the following MWE, which produces the crash without requiring anything from FlexiBLAS.

About this, it seems that, with gcc 12, when FFLAGS contains -flto=auto, the generated VerifyFortran.h contains nothing, thus the test error. There's a warning before that:

CMake Warning (dev) at /usr/share/cmake/Modules/FortranCInterface.cmake:309 (message):
  No FortranCInterface mangling known for VerifyFortran
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FortranCInterface/Verify/CMakeLists.txt:16 (FortranCInterface_HEADER)
This warning is for project developers.  Use -Wno-dev to suppress it.

Any idea about how to debug this, how to tell cmake to output every command it is running?

grisuthedragon commented 2 years ago

There are some flags like --debug-trycompile and --loglevel. This cause more information in CMakeFiles/CMakeOutput.log and CMakeFiles/CMakeError.log but I also never managed to get cmake everything. In general this seems to an issue for gcc and/or cmake.