brainvisa / casa-distro

Unified development environment for BrainVISA projects.
2 stars 1 forks source link

Test building a casa-distro environment in a Conda environment #345

Open sapetnioc opened 1 year ago

sapetnioc commented 1 year ago

Conda is a packaging system that is widely used in research (e.g. FSL is now distributed through Conda). At first sight, it seems to provide a complete self-content environment that is not limited to Python-related packages. If we succeed in building a casa-distro environment based on a Conda environment, it could make it possible to have a fully modular environment that would be highly compatible with a standard packaging system of the Python community.

The first thing to do could be to start from a Miniforge environment and build our software on the top of it without using a container. If it is successful, we would have the possibility to build conda package for a specific system and to bring Apptainer in the game as a minimal read-only base system using a read-write conda environment.

sapetnioc commented 1 year ago

I have been able to build an environment from scratch (using an AppTainer container) and install dependencies mainly with Mamba (C++ implementation of Conda packaging system). Not counting sources download, the whole process to get ready to start bv_maker takes 4 minute on my laptop (it was 8 minutes before I switch to Mamba). There are some packages missing (tensorflow for instance) but is it fast anyway.

I am stuck with PyQt configuration in CMake since there is no sip executable. There is a sip package that is meant to build PyQt extensions using *.sip files but it does not provide the command. Instead, it follows PEP 517 and PEP 518 specifications that defines a build system for Python. I found a few information in this page. I do not know yet if it will be possible to adapt our compilation code to this system.

It is possible to test my very experimental build system. There is a recipe here.

denisri commented 1 year ago

In sip >= 6, the sip command is not sip any longer, but is replaced with several ones, sip-build, sip-distinfo, sip-install, sip-module, sip-sdist, sip-wheel. Outside a conda environment, they are installed via the pip package sip, but I don't know if they are packages the same way in condaforge. I have started adaptations of the build system for sip6/Qt6 in the experimental containers casa-dev-5.4, based on the pip packages. I think sip6 can also build for Qt5/PyQt5 (I have not tried, I wanted to port to Qt6/PyQt6), and the builds seem to work provided changes in LD_LIBRARY_PATH to use the Qt libs in the pip packages at runtime. But this setup is still hybrid because it builds C++ sources with the system Qt6, and sip bindings with the pip install of Qt6. Anyway I have done the adaptations for both sip6 and Qt6 and have assumed using sip6 for Qt6, thus in bv_maker.cfg you switch to it using cmake_options += -DDESIRED_QT_VERSION=6. Maybe using this in your new dev environment would be a path to the solution. But the other critical point then is: could you use the Qt libs and includes installed in conda with cmake/bv_maker for C++ code (not speaking of sip and PyQt) ? If yes I guess we are close to the solution...

sapetnioc commented 1 year ago

I am pleased to hear that because these sip-* commands are installed. I cannot answer your question yet but what I know is encouraging:

denisri commented 1 year ago

This looks good... except that bv_maker handles sip6 with Qt6 only for now.

sapetnioc commented 1 year ago

I do not see PyQt6 packaged in conda-forge but there are older sip packages. I installed version 4.19.25 that installed. Now I have to find a way to install pyqt without upgrading sip, there is a pyqt5-sip package, let's try.

denisri commented 1 year ago

There was a bug in sip 4.19.x which caused crashes in our programs. The bug has been fixed, but not in the 4.x branch which is considered obsolete by the author. That's why we had to rebuild and reinstall a version 4.19.15 (prior to the bug), and rebuild PyQt using it, in our apptainer images. So sip4 packages will certainly cause problems. Plus they are probably not compatible with recent PyQt binary distributions. At this point we should use sip6, and either jump to Qt6, or finish adapting bv_maker to build with Qt5 and sip6.

sapetnioc commented 1 year ago

Well, after some fighting, I had been able to finish a bv_maker configure and to start a compilation.

denisri commented 1 year ago

Great ! Using which sip ?

sapetnioc commented 1 year ago

The bad one. But I am stuck on a compilation error on aims-free that may be due to a change in gcc (12.2.0 in conda-forge). I am writing an issue in aims-free and I am also rewriting the scripts to download end build everything (and get the error). I tried to download the source separately with git submodules but it is not appropriate, I will just make a script for now.

sapetnioc commented 1 year ago

Anatomis is being compiled ! There will be problems due to sip but this is very promising. And the time to be able to launch bv_maker from scratch is less than 4 minutes in Neurospin. It seems that network bandwidth plays an important role here since it took me more than 20 minutes this week-end with a slower connection (but same laptop).

denisri commented 1 year ago

So we have to distinguish between sip6 and Qt6 in brainvisa-cmake in order to build for sip6+Qt5.

denisri commented 1 year ago

I have pushed an experimental change to allow to switch to sip6 while using Qt5. In bv_maker.cfg, add the line:

  cmake_options += -DDESIRED_QT_VERSION=5 -DDESIRED_SIP_VERSION=6

I have not tested beyond the selection of sip6 and PyQt5 in the CMakeCache.txt because I don't have a conda environment yet, and pip doesn't provide dev headers for Qt (but it's the same situation for PyQt6 so I could build and test a mixed-up build using the system Qt5 and runtime of PyQt5).

sapetnioc commented 1 year ago

I will give a try to your solution for sip 6. However, I am stuck on a problem that may be difficult to solve. There is one library that I had to install with apt and not from conda-forge: libGL.so. This causes two problems:

  1. CMake cannot find the library. I do not know exactly why yet but this is something we can handle.
  2. If I copy/paste the linking command and add manually the libraries missed by CMake (libGL.so.1 and libGLX.so.0), I get undefined symbols both from OpenGL and glibc :
Apptainer> cd /casa/build/build_files/aims-free/aimsgui/src
Apptainer> /casa/conda/bin/c++ -O3 -DNDEBUG CMakeFiles/AimsAttributedViewer.dir/AimsAttributedViewer/attributedViewer.cc.o -o ../../../../bin/AimsAttributedViewer  -Wl,-rpath,/casa/build/lib: ../../../../lib/libaimsgui.so.5.2.0 /casa/conda/lib/libQt5Widgets.so.5.15.8 /casa/conda/lib/libQt5Gui.so.5.15.8 /casa/conda/lib/libQt5Sql.so.5.15.8 /casa/conda/lib/libQt5Core.so.5.15.8 ../../../../lib/libaimsdata.so.5.2.0 ../../../../lib/libgraph.so.5.2.0 ../../../../lib/libcartodata.so.5.2.0 ../../../../lib/libsoma-io.so.5.2.0 /usr/lib/x86_64-linux-gnu/libGL.so.1 /usr/lib/x86_64-linux-gnu/libGLX.so.0 -ldl ../../../../lib/libcartobase.so.5.2.0 ../../../../lib/libcartobase_1.so.5.2.0 ../../../../lib/libcartobase_0.so.5.2.0 /casa/conda/lib/libxml2.so /casa/conda/lib/libsigc-2.0.so -lm -lpthread
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libGLdispatch.so.0, needed by /usr/lib/x86_64-linux-gnu/libGL.so.1, not found (try using -rpath or -rpath-link)
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchDestroyTable'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchGetABIVersion'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchCreateTable'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGL.so.1: undefined reference to `__glDispatchFini'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `dlclose@GLIBC_2.34'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGL.so.1: undefined reference to `__glDispatchRegisterStubCallbacks'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchLoseCurrent'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `dlopen@GLIBC_2.34'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /casa/conda/lib/libQt5Widgets.so.5.15.8: undefined reference to `memcpy@GLIBC_2.14'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGL.so.1: undefined reference to `__glDispatchUnregisterStubCallbacks'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchNewVendorID'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /casa/conda/lib/libQt5Core.so.5.15.8: undefined reference to `getauxval@GLIBC_2.16'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchForceUnpatch'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGL.so.1: undefined reference to `_glapi_tls_Current'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchGetCurrentThreadState'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchCheckMultithreaded'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /casa/conda/lib/libQt5Core.so.5.15.8: undefined reference to `clock_gettime@GLIBC_2.17'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `dlsym@GLIBC_2.34'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchMakeCurrent'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGL.so.1: undefined reference to `__glDispatchInit'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchGetProcAddress'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so.0: undefined reference to `__glDispatchReset'
collect2: error: ld returned 1 exit status

For glibc, I will try to find-out what kind of system is used to build conda-forge packages (I suspect it is CentOS 6 or 7). Using a system with a more recent Ubuntu may solve some glibc problems. There are so few system dependencies (wget, build-essential, distro-info-data and libgl1-mesa-dev) that it is not impossible to switch to a non Debian system if it is more compatible with conda-forge.

denisri commented 1 year ago

For sip6 at least in pip installs of PyQt5, there is a mismatch in sip ABI versions of the module lib and includes (well I'm not sure yet of what happens, I'm still struggling with it). So if it doesn't work straight for you, hold on...

For libGL, it is needed by Qt5 (libQt5Gui.so links against OpenGL) so the library has to be present on the system anyway. "Modern" versions of libGL on linux are using a switch library called "gldispatch", which goal is to redirect GL calls to the correct implementation (hardware, software, nvidia...). I guess you also have to link against it if you are linking with a libGL which implements this switch mechanism. But CMake should be able to handle it, as we are using it in our libraries for many years. You are probably missing a dev package. On Ubuntui 22.04 we have libgl-dev, libglx-dev, libglu1-mesa-dev. libGLdispatch.so is part of the package libglvnd0, and should be installed as a dependency of libgl I guess.

denisri commented 1 year ago

I can't make it work using pip installs. There seems to be a discrepancy between the sip runtime module and the sip-build tool. I can compile pyaims but not run it, it fails with a sip ABI mismatch error. By the way I have discovered the existence of sip5 which doesn't appear on riverbankcomputing site... and it did not work any better. I'm giving up. Hope you're more lucky using conda.

sapetnioc commented 1 year ago

Well, not really. On the OpenGL side I still have undefined symbols (I got rid of glibc symbols using CentOS 7 but got other problems) but as you can see in CMake log below, I can try to add option to use GLVND. I can also try your suggestion to find a gldispatch library. On the sip side, I switched back to sip 6 merged your mods but there are two errors in bv_maker configure:

=== now configuring all other projects ===
$ cmake -DDESIRED_QT_VERSION=5 -DDESIRED_SIP_VERSION=6 -DCMAKE_BUILD_TYPE:STRING=Release -DBRAINVISA_CMAKE_BUILD_TYPE=no-brainvisa-cmake /casa/build
-- Add specific paths for Ubuntu distribution
-- Configuring component brainvisa-cmake from source directory "/casa/src/brainvisa-cmake"
-- Configuring component casa-distro from source directory "/casa/build/build_files/casa-distro_src"
-- Configuring component brainvisa-share from source directory "/casa/src/brainvisa-share"
-- Found PythonInterp: /casa/conda/bin/python3 (Required is at least version "2") 
-- Configuring component soma-base from source directory "/casa/src/soma-base"
-- Configuring component soma-io from source directory "/casa/src/soma-io"
-- Checking for one of the modules 'openslide'
-- Configuring component soma-workflow from source directory "/casa/src/soma-workflow"
-- LibDRMAA was not found.
-- Configuring component capsul from source directory "/casa/build/build_files/capsul_src"
-- Configuring component populse_db from source directory "/casa/build/build_files/populse_db_src"
-- Configuring component aims-free from source directory "/casa/src/aims-free"
-- Checking for one of the modules 'openslide'
-- Found PythonInterp: /casa/conda/bin/python3  
CMake Deprecation Warning at /casa/src/aims-free/aimsgui/src/CiftiLib-1.5.1/CMakeLists.txt:7 (CMAKE_POLICY):
  The OLD behavior for policy CMP0045 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.

RUN MAKETEMPLATES: /casa/conda/bin/python3;/casa/build/share/aims-5.2/sip/maketemplates.py;-i;/casa/src/aims-free/pyaims/src/sip/generatedtypes.py;-p;/casa/src/aims-free/pyaims/src/sip;-o;/casa/build/share/aims-5.2/sip;-t;/casa/src/aims-free/pyaims/src/sip/typessub.py;-l;-m;/casa/conda/bin/moc;-P;cpp -C;-D;SIP_MODULE="PyQt5.sip"
Traceback (most recent call last):
  File "/casa/build/share/aims-5.2/sip/maketemplates.py", line 159, in <module>
    six.exec_(code, globals(), globals())
  File "/casa/src/aims-free/pyaims/src/sip/generatedtypes.py", line 36, in <module>
    import sip
ModuleNotFoundError: No module named 'sip'
CMake Error at /casa/src/aims-free/pyaims/src/sip/CMakeLists.txt:48 (message):
  The sip generation from pyaims templates failed.  Command:

  returned output:

  ========================

Traceback (most recent call last):
  File "/casa/build/share/aims-5.2/sip/maketemplates.py", line 172, in <module>
    six.exec_(code, globals(), globals())
  File "/casa/src/aims-free/pyaims/src/sip/typessub.py", line 37, in <module>
    import sip
ModuleNotFoundError: No module named 'sip'
CMake Error at /casa/src/aims-free/pyaimsalgo/src/sip/CMakeLists.txt:27 (message):
  The sip generation from pyaimsalgo templates with
  "/casa/build/share/aims-5.2/sip/maketemplates.py" returned an error:

-- Configuring component aims-gpl from source directory "/casa/src/aims-gpl"
-- Checking for one of the modules 'openslide'
Traceback (most recent call last):
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 110, in <module>
    sip_dir = get_default_sip_dir()
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 44, in get_default_sip_dir
    import sipconfig
ModuleNotFoundError: No module named 'sipconfig'
CMake Error at /casa/src/aims-gpl/pyaimsgui/src/sip_gui/CMakeLists.txt:28 (message):
  The sip generation from pyaimsgui templates with
  "/casa/build/share/aims-5.2/sip/maketemplates.py" returned an error:

  Traceback (most recent call last):

    File "/casa/build/share/aims-5.2/sip/maketemplates.py", line 172, in <module>
      six.exec_(code, globals(), globals())
    File "/casa/build/share/aims-5.2/sip/typessub.py", line 37, in <module>
      import sip

  ModuleNotFoundError: No module named 'sip'

  ========================

Warning: file "/casa/src/aims-gpl/pyaimsgui/src/sip_gui/Traceback (most recent call last):
  File "/casa/build/share/aims-5.2/sip/maketemplates.py", line 172, in <module>
    six.exec_(code, globals(), globals())
  File "/casa/build/share/aims-5.2/sip/typessub.py", line 37, in <module>
    import sip
ModuleNotFoundError: No module named 'sip'" does not exists
-- Configuring component anatomist-free from source directory "/casa/src/anatomist-free"
-- Checking for one of the modules 'openslide'
CMake Warning (dev) at /casa/conda/share/cmake-3.26/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  /casa/src/anatomist-free/CMakeLists.txt:8 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Qwt was not found.
VTK5 and Qt5+ are not supported. The VTK plugin is disabled.
WARNING: anatomist modules anaroibase and anahistogram will not be build because Qwt library has not been found
WARNING: anatomist modules anavtk will not be build because VTK or vtkINRIA3D library has not been found
-- Configuring component anatomist-gpl from source directory "/casa/src/anatomist-gpl"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
CMake Warning (dev) at /casa/conda/share/cmake-3.26/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  share/anatomist-free-5.2/cmake/anatomist-free-config.cmake:65 (find_package)
  share/brainvisa-cmake-5.2/cmake/brainvisa-cmake-config.cmake:259 (find_package)
  /casa/src/anatomist-gpl/CMakeLists.txt:6 (BRAINVISA_FIND_PACKAGE)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Qwt was not found.
Traceback (most recent call last):
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 110, in <module>
    sip_dir = get_default_sip_dir()
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 44, in get_default_sip_dir
    import sipconfig
ModuleNotFoundError: No module named 'sipconfig'
WARNING: Not building pyanatomist due to missing dependency:
  PYTHON_FOUND=TRUE
SIP_FOUND=TRUE
NUMPY_FOUND=TRUE
PYQT5_FOUND=

-- Configuring component axon from source directory "/casa/src/axon"
-- Checking for one of the modules 'openslide'
Traceback (most recent call last):
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 110, in <module>
    sip_dir = get_default_sip_dir()
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 44, in get_default_sip_dir
    import sipconfig
ModuleNotFoundError: No module named 'sipconfig'
SIP VERSION: 6.7.9
-- Configuring component brainvisa-spm from source directory "/casa/src/brainvisa-spm"
-- Configuring component highres-cortex from source directory "/casa/src/highres-cortex"
-- Checking for one of the modules 'openslide'
-- Found OpenMP: TRUE (found version "4.5")  
-- Configuring component morphologist-nonfree from source directory "/casa/src/morphologist-nonfree"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
-- Configuring component morphologist-gpl from source directory "/casa/src/morphologist-gpl"
-- Checking for one of the modules 'openslide'
-- Configuring component morphologist-baby from source directory "/casa/src/morphologist-baby"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
-- Configuring component sulci-nonfree from source directory "/casa/src/sulci-nonfree"
-- Checking for one of the modules 'openslide'
-- Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR) 
Traceback (most recent call last):
  File "/casa/build/share/aims-5.2/sip/maketemplates.py", line 172, in <module>
    six.exec_(code, globals(), globals())
  File "/casa/build/share/aims-5.2/sip/typessub.py", line 37, in <module>
    import sip
ModuleNotFoundError: No module named 'sip'
CMake Error at /casa/src/sulci-nonfree/pysigraph/src/sip/CMakeLists.txt:29 (message):
  The sip generation from sulci-nonfree templates with
  "/casa/build/share/aims-5.2/sip/maketemplates.py" returned an error:

  ========================

-- Checking for one of the modules 'openslide'
CMake Warning (dev) at /casa/conda/share/cmake-3.26/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  share/anatomist-free-5.2/cmake/anatomist-free-config.cmake:65 (find_package)
  share/brainvisa-cmake-5.2/cmake/brainvisa-cmake-config.cmake:259 (find_package)
  /casa/src/sulci-nonfree/CMakeLists.txt:35 (BRAINVISA_FIND_PACKAGE)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Qwt was not found.
-- Configuring component morphologist-ui from source directory "/casa/src/morphologist"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
CMake Warning (dev) at /casa/conda/share/cmake-3.26/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  share/anatomist-free-5.2/cmake/anatomist-free-config.cmake:65 (find_package)
  share/brainvisa-cmake-5.2/cmake/brainvisa-cmake-config.cmake:259 (find_package)
  share/anatomist-gpl-5.2/cmake/anatomist-gpl-config.cmake:36 (BRAINVISA_FIND_PACKAGE)
  share/brainvisa-cmake-5.2/cmake/brainvisa-cmake-config.cmake:259 (find_package)
  /casa/src/morphologist/CMakeLists.txt:7 (BRAINVISA_FIND_PACKAGE)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Qwt was not found.
-- Configuring component morpho-deepsulci from source directory "/casa/src/morpho-deepsulci"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
-- Configuring component brainrat-gpl from source directory "/casa/src/brainrat-gpl"
-- Checking for one of the modules 'openslide'
-- Configuring component brainrat-private from source directory "/casa/src/brainrat-private"
-- Checking for one of the modules 'openslide'
-- Configuring component bioprocessing from source directory "/casa/src/bioprocessing"
-- Found OpenMP: TRUE (found version "4.5")  
-- Checking for one of the modules 'openslide'
-- Configuring component preclinical-imaging-iam from source directory "/casa/src/preclinical-imaging-iam"
-- Configuring component primatologist-gpl from source directory "/casa/src/primatologist-gpl"
-- Checking for one of the modules 'openslide'
-- Configuring component 3dns-private from source directory "/casa/src/3dns-private"
CMake Deprecation Warning at /casa/src/3dns-private/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- Checking for one of the modules 'openslide'
-- Configuring component constellation-gpl from source directory "/casa/src/constellation-gpl"
-- Configuring component constellation-nonfree from source directory "/casa/src/constellation-nonfree"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
CMake Warning (dev) at /casa/conda/share/cmake-3.26/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  share/anatomist-free-5.2/cmake/anatomist-free-config.cmake:65 (find_package)
  /casa/src/constellation-nonfree/CMakeLists.txt:24 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Qwt was not found.
-- Configuring component cortical_surface-nonfree from source directory "/casa/src/cortical_surface-nonfree"
-- Checking for one of the modules 'openslide'
-- Configuring component cortical_surface-gpl from source directory "/casa/src/cortical_surface-gpl"
-- Configuring component brainvisa_freesurfer from source directory "/casa/src/brainvisa_freesurfer"
Traceback (most recent call last):
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 110, in <module>
    sip_dir = get_default_sip_dir()
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 44, in get_default_sip_dir
    import sipconfig
ModuleNotFoundError: No module named 'sipconfig'
-- Configuring component nuclear_imaging-gpl from source directory "/casa/src/nuclear_imaging-gpl"
-- Configuring component nuclear_imaging-nonfree from source directory "/casa/src/nuclear_imaging-nonfree"
-- Checking for one of the modules 'openslide'
-- Configuring component catidb-client from source directory "/casa/build/build_files/catidb-client_src"
-- Configuring component sacha-nonfree from source directory "/casa/src/sacha-nonfree"
-- Checking for one of the modules 'openslide'
-- Configuring component sacha-gpl from source directory "/casa/src/sacha-gpl"
-- Configuring component whasa-nonfree from source directory "/casa/src/whasa-nonfree"
-- Checking for one of the modules 'openslide'
-- Configuring component whasa-gpl from source directory "/casa/src/whasa-gpl"
-- Configuring component longitudinal_pipelines from source directory "/casa/src/longitudinal_pipelines"
-- Checking for one of the modules 'openslide'
-- Configuring component disco from source directory "/casa/src/brainvisa-disco"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
CMake Warning (dev) at /casa/conda/share/cmake-3.26/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  share/anatomist-free-5.2/cmake/anatomist-free-config.cmake:65 (find_package)
  share/brainvisa-cmake-5.2/cmake/brainvisa-cmake-config.cmake:259 (find_package)
  /casa/src/brainvisa-disco/CMakeLists.txt:45 (BRAINVISA_FIND_PACKAGE)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Qwt was not found.
-- Configuring component qualicati from source directory "/casa/build/build_files/qualicati_src"
-- Configuring component deidentification from source directory "/casa/build/build_files/deidentification_src"
-- Configuring component rsfmri from source directory "/casa/build/build_files/rsfmri_src"
-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
  File "/casa/build/python/brainvisa_cmake/commands.py", line 68, in process
    logs = getattr(o, method)(*meth_args,
  File "/casa/build/python/brainvisa_cmake/build.py", line 606, in configure
    system(cwd=self.directory,
  File "/casa/build/python/brainvisa_cmake/subprocess.py", line 59, in system
    six.reraise(*error)
  File "/casa/conda/lib/python3.10/site-packages/six.py", line 718, in reraise
    raise value.with_traceback(tb)
  File "/casa/build/python/brainvisa_cmake/subprocess.py", line 48, in system
    raise OSError()
OSError: Command failed: 'cmake' '-DDESIRED_QT_VERSION=5' '-DDESIRED_SIP_VERSION=6' '-DCMAKE_BUILD_TYPE:STRING=Release' '-DBRAINVISA_CMAKE_BUILD_TYPE=no-brainvisa-cmake' '/casa/build'
From directory: /casa/build
sapetnioc commented 1 year ago

I think the FindSIP.cmake problem was due to brainvisa-cmake that was run from build tree and not correctly updated.

denisri commented 1 year ago

However the direct import of sip in generatedtypes.py is an error since now the main sip module may be different from PyQt5.sip or PyQt6.sip. I have to fix it also. I confess I don't follow the problems with opengl. Can't you build using the system opengl like we do for all brainvisa distributions ?

denisri commented 1 year ago

For sip I have pushed a fix: brainvisa/aims-free@5a33f08ad2f0df59504d06327ce788cdd18f4a01.

sapetnioc commented 1 year ago

Thanks, there are still errors due to the lack of sipconfig module:

RUN MAKETEMPLATES: /casa/conda/bin/python3;/casa/build/share/aims-5.2/sip/maketemplates.py;-i;/casa/src/aims-free/pyaims/src/sip/generatedtypes.py;-p;/casa/src/aims-free/pyaims/src/sip;-o;/casa/build/share/aims-5.2/sip;-t;/casa/src/aims-free/pyaims/src/sip/typessub.py;-l;-m;/casa/conda/bin/moc;-P;cpp -C;-D;SIP_MODULE="PyQt5.sip"
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:55:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:55:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:55:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:67:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:67:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:57:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:51:5: warning: character constant too long for its type
<stdin>:10:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
<stdin>:2:5: warning: character constant too long for its type
error in generation of /casa/build/share/aims-5.2/sip/distancemap_S16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/distancemap_FLOAT.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/diffusionsmoother_S16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/diffusionsmoother_FLOAT.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/gradient_FLOAT.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/meshresampler_rc_ptr_Volume_U32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/meshresampler_BucketMap_VOID.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/moment_U8.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/moment_S16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/moment_U16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/moment_S32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/moment_U32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/moment_FLOAT.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/moment_DOUBLE.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/morphomath_U8.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/morphomath_S16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/morphomath_U16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/morphomath_S32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/morphomath_U32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/morphomath_FLOAT.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/morphomath_DOUBLE.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/polynomial_FLOAT_3.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_U8.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_S16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_U16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_S32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_U32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_FLOAT.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_DOUBLE.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_RGB.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_RGBA.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_HSV.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/resampler_POINT3DF.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/sampler_FLOAT_3.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/smoothing_U8.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/smoothing_S16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/smoothing_U16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/smoothing_S32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/smoothing_U32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/smoothing_FLOAT.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/smoothing_DOUBLE.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/histogram_U8.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/histogram_S16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/histogram_U16.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/histogram_S32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/histogram_U32.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/histogram_FLOAT.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/histogram_DOUBLE.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/rcptr_FfdTransformation.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/rcptr_SplineFfd.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/rcptr_TrilinearFfd.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/rcptr_GeometricProperties.sip :
No module named 'sipconfig'
CMake Error at /casa/src/aims-free/pyaimsalgo/src/sip/CMakeLists.txt:27 (message):
  The sip generation from pyaimsalgo templates with
  "/casa/build/share/aims-5.2/sip/maketemplates.py" returned an error:

  /casa/build/share/aims-5.2/sip/distancemap_S16.sip;/casa/build/share/aims-5.2/sip/distancemap_FLOAT.sip;/casa/build/share/aims-5.2/sip/diffusionsmoother_S16.sip;/casa/build/share/aims-5.2/sip/diffusionsmoother_FLOAT.sip;/casa/build/share/aims-5.2/sip/gradient_FLOAT.sip;/casa/build/share/aims-5.2/sip/meshresampler_rc_ptr_Volume_U32.sip;/casa/build/share/aims-5.2/sip/meshresampler_BucketMap_VOID.sip;/casa/build/share/aims-5.2/sip/moment_U8.sip;/casa/build/share/aims-5.2/sip/moment_S16.sip;/casa/build/share/aims-5.2/sip/moment_U16.sip;/casa/build/share/aims-5.2/sip/moment_S32.sip;/casa/build/share/aims-5.2/sip/moment_U32.sip;/casa/build/share/aims-5.2/sip/moment_FLOAT.sip;/casa/build/share/aims-5.2/sip/moment_DOUBLE.sip;/casa/build/share/aims-5.2/sip/morphomath_U8.sip;/casa/build/share/aims-5.2/sip/morphomath_S16.sip;/casa/build/share/aims-5.2/sip/morphomath_U16.sip;/casa/build/share/aims-5.2/sip/morphomath_S32.sip;/casa/build/share/aims-5.2/sip/morphomath_U32.sip;/casa/build/share/aims-5.2/sip/morphomath_FLOAT.sip;/casa/build/share/aims-5.2/sip/morphomath_DOUBLE.sip;/casa/build/share/aims-5.2/sip/polynomial_FLOAT_3.sip;/casa/build/share/aims-5.2/sip/resampler_U8.sip;/casa/build/share/aims-5.2/sip/resampler_S16.sip;/casa/build/share/aims-5.2/sip/resampler_U16.sip;/casa/build/share/aims-5.2/sip/resampler_S32.sip;/casa/build/share/aims-5.2/sip/resampler_U32.sip;/casa/build/share/aims-5.2/sip/resampler_FLOAT.sip;/casa/build/share/aims-5.2/sip/resampler_DOUBLE.sip;/casa/build/share/aims-5.2/sip/resampler_RGB.sip;/casa/build/share/aims-5.2/sip/resampler_RGBA.sip;/casa/build/share/aims-5.2/sip/resampler_HSV.sip;/casa/build/share/aims-5.2/sip/resampler_POINT3DF.sip;/casa/build/share/aims-5.2/sip/sampler_FLOAT_3.sip;/casa/build/share/aims-5.2/sip/smoothing_U8.sip;/casa/build/share/aims-5.2/sip/smoothing_S16.sip;/casa/build/share/aims-5.2/sip/smoothing_U16.sip;/casa/build/share/aims-5.2/sip/smoothing_S32.sip;/casa/build/share/aims-5.2/sip/smoothing_U32.sip;/casa/build/share/aims-5.2/sip/smoothing_FLOAT.sip;/casa/build/share/aims-5.2/sip/smoothing_DOUBLE.sip;/casa/build/share/aims-5.2/sip/histogram_U8.sip;/casa/build/share/aims-5.2/sip/histogram_S16.sip;/casa/build/share/aims-5.2/sip/histogram_U16.sip;/casa/build/share/aims-5.2/sip/histogram_S32.sip;/casa/build/share/aims-5.2/sip/histogram_U32.sip;/casa/build/share/aims-5.2/sip/histogram_FLOAT.sip;/casa/build/share/aims-5.2/sip/histogram_DOUBLE.sip;/casa/build/share/aims-5.2/sip/rcptr_FfdTransformation.sip;/casa/build/share/aims-5.2/sip/rcptr_SplineFfd.sip;/casa/build/share/aims-5.2/sip/rcptr_TrilinearFfd.sip;/casa/build/share/aims-5.2/sip/rcptr_GeometricProperties.sip

Warning: file "/casa/build/share/aims-5.2/sip/rcptr_GeometricProperties.sip
" does not exists
-- Configuring component aims-gpl from source directory "/casa/src/aims-gpl"
-- Checking for one of the modules 'openslide'
Traceback (most recent call last):
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 110, in <module>
    sip_dir = get_default_sip_dir()
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 44, in get_default_sip_dir
    import sipconfig
ModuleNotFoundError: No module named 'sipconfig'
CMake Error at /casa/src/aims-gpl/pyaimsgui/src/sip_gui/CMakeLists.txt:28 (message):
  The sip generation from pyaimsgui templates with
  "/casa/build/share/aims-5.2/sip/maketemplates.py" returned an error:

  error in generation of /casa/build/share/aims-5.2/sip/aimsgui_VOID.sip :

  No module named 'sipconfig'

  error in generation of
  /casa/build/share/aims-5.2/sip/list_CurrentGraphData.sip :

  No module named 'sipconfig'

  /casa/build/share/aims-5.2/sip/aimsgui_VOID.sip;/casa/build/share/aims-5.2/sip/list_CurrentGraphData.sip

  ========================

Warning: file "/casa/src/aims-gpl/pyaimsgui/src/sip_gui/error in generation of /casa/build/share/aims-5.2/sip/aimsgui_VOID.sip :
No module named 'sipconfig'
error in generation of /casa/build/share/aims-5.2/sip/list_CurrentGraphData.sip :
No module named 'sipconfig'
/casa/build/share/aims-5.2/sip/aimsgui_VOID.sip" does not exists
-- Configuring component anatomist-free from source directory "/casa/src/anatomist-free"
-- Checking for one of the modules 'openslide'
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so   
-- Qwt was not found.
VTK5 and Qt5+ are not supported. The VTK plugin is disabled.
WARNING: anatomist modules anaroibase and anahistogram will not be build because Qwt library has not been found
WARNING: anatomist modules anavtk will not be build because VTK or vtkINRIA3D library has not been found
-- Configuring component anatomist-gpl from source directory "/casa/src/anatomist-gpl"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
-- Qwt was not found.
Traceback (most recent call last):
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 110, in <module>
    sip_dir = get_default_sip_dir()
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 44, in get_default_sip_dir
    import sipconfig
ModuleNotFoundError: No module named 'sipconfig'
WARNING: Not building pyanatomist due to missing dependency:
  PYTHON_FOUND=TRUE
SIP_FOUND=TRUE
NUMPY_FOUND=TRUE
PYQT5_FOUND=

-- Configuring component axon from source directory "/casa/src/axon"
-- Checking for one of the modules 'openslide'
Traceback (most recent call last):
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 110, in <module>
    sip_dir = get_default_sip_dir()
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 44, in get_default_sip_dir
    import sipconfig
ModuleNotFoundError: No module named 'sipconfig'
SIP VERSION: 6.7.9
-- Configuring component brainvisa-spm from source directory "/casa/src/brainvisa-spm"
-- Configuring component highres-cortex from source directory "/casa/src/highres-cortex"
-- Checking for one of the modules 'openslide'
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Configuring component morphologist-nonfree from source directory "/casa/src/morphologist-nonfree"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
-- Configuring component morphologist-gpl from source directory "/casa/src/morphologist-gpl"
-- Checking for one of the modules 'openslide'
-- Configuring component morphologist-baby from source directory "/casa/src/morphologist-baby"
-- Checking for one of the modules 'openslide'
-- Checking for one of the modules 'openslide'
-- Configuring component sulci-nonfree from source directory "/casa/src/sulci-nonfree"
-- Checking for one of the modules 'openslide'
-- -- Found LibSVM2: /casa/conda/lib/libsvm.so (found version: "3.31")
-- Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR) 
makeTemplate, without moc: None
error in generation of /casa/build/share/sulci-nonfree-5.2/sip/set_AdaptivePtr.sip :
No module named 'sipconfig'
makeTemplate, without moc: None
error in generation of /casa/build/share/sulci-nonfree-5.2/sip/set_CliquePtr.sip :
No module named 'sipconfig'
makeTemplate, without moc: None
error in generation of /casa/build/share/sulci-nonfree-5.2/sip/set_rc_ptr_Clique.sip :
No module named 'sipconfig'
makeTemplate, without moc: None
error in generation of /casa/build/share/sulci-nonfree-5.2/sip/rcptr_Clique.sip :
No module named 'sipconfig'
CMake Error at /casa/src/sulci-nonfree/pysigraph/src/sip/CMakeLists.txt:29 (message):
  The sip generation from sulci-nonfree templates with
  "/casa/build/share/aims-5.2/sip/maketemplates.py" returned an error:

  /casa/build/share/sulci-nonfree-5.2/sip/set_AdaptivePtr.sip;/casa/build/share/sulci-nonfree-5.2/sip/set_CliquePtr.sip;/casa/build/share/sulci-nonfree-5.2/sip/set_rc_ptr_Clique.sip;/casa/build/share/sulci-nonfree-5.2/sip/rcptr_Clique.sip

  ========================

Warning: file "/casa/build/share/sulci-nonfree-5.2/sip/rcptr_Clique.sip
" does not exists
-- Checking for one of the modules 'openslide'
denisri commented 1 year ago

oh yes, other projects bindings needed to be adapted too... it's done.

sapetnioc commented 1 year ago

I still have errors. I try to fix them myself but you may already know how to find the default sip dir without sipconfig. And I have difficulties to find where this yQt5 comes from.

Warning: file "/casa/build/share/aims-5.2/sip/rcptr_GeometricProperties.sip
" does not exists
-- Configuring component aims-gpl from source directory "/casa/src/aims-gpl"
-- Checking for one of the modules 'openslide'
Traceback (most recent call last):
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 110, in <module>
    sip_dir = get_default_sip_dir()
  File "/casa/build/share/brainvisa-cmake-5.2/cmake/modules/FindPyQt.py", line 44, in get_default_sip_dir
    import sipconfig
ModuleNotFoundError: No module named 'sipconfig'
CMake Error at /casa/src/aims-gpl/pyaimsgui/src/sip_gui/CMakeLists.txt:33 (message):
  The sip generation from pyaimsgui templates with
  "/casa/build/share/aims-5.2/sip/maketemplates.py" returned an error:

  Traceback (most recent call last):

    File "/casa/build/share/aims-5.2/sip/maketemplates.py", line 108, in <module>
      sip = importlib.import_module(sip_mod)
    File "/casa/conda/lib/python3.10/importlib/__init__.py", line 126, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
    File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
    File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
    File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
    File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
    File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked

  ModuleNotFoundError: No module named 'yQt5'

  ========================
denisri commented 1 year ago

Well, sometimes the sip module name is between quotes, and sometimes not - I don't know why but we can handle both cases. I have pushed another fix.

sapetnioc commented 1 year ago

Ok, configuration is ok with PyQt 5/sip 6. Compilation is ongoing. I will probably have to deal with OpenGL.

I made a change in aims sip generation that should have been reviewed before push (too late). Can you give me your feeling @denisri : https://github.com/brainvisa/aims-free/commit/858c663e28a38fb265f181b59c995a4f66eb7194#

denisri commented 1 year ago

I have pushed changes similar to the first ones in your code, it's strange it did not cause conflicts? Otherwise I think it is OK for sip >= 4.19.15.

sapetnioc commented 1 year ago

Most of the code was compiled. Now I am still stuck by the OpenGL/glibc linking problem. conda-forge comes with two glibc ABI : 2.12 (the default) and 2.17. Qt is compiled with 2.17, this is why I had errors with both Qt (from Conda) and OpenGL (from system). Switching to 2.17 (so easy by the way, just mamba install sysroot_linux-64=2.17) solves the Qt part. There is still the OpenGL problems:

Apptainer> /casa/conda/bin/c++ -O3 -L/usr/lib/x86_64-linux-gnu -DNDEBUG CMakeFiles/AimsAttributedViewer.dir/AimsAttributedViewer/attributedViewer.cc.o -o ../../../../bin/AimsAttributedViewer  -Wl,-rpath,/casa/build/lib: ../../../../lib/libaimsgui.so.5.2.0 /casa/conda/lib/libQt5Widgets.so.5.15.8 /casa/conda/lib/libQt5Gui.so.5.15.8 /casa/conda/lib/libQt5Sql.so.5.15.8 /casa/conda/lib/libQt5Core.so.5.15.8 ../../../../lib/libaimsdata.so.5.2.0 ../../../../lib/libgraph.so.5.2.0 ../../../../lib/libcartodata.so.5.2.0 ../../../../lib/libsoma-io.so.5.2.0 -ldl ../../../../lib/libcartobase.so.5.2.0 ../../../../lib/libcartobase_1.so.5.2.0 ../../../../lib/libcartobase_0.so.5.2.0 /casa/conda/lib/libxml2.so /casa/conda/lib/libsigc-2.0.so -lm -lpthread -lGL -lGLX -lGLdispatch
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so: undefined reference to `dlclose@GLIBC_2.34'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so: undefined reference to `dlopen@GLIBC_2.34'
/casa/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/12.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/lib/x86_64-linux-gnu/libGLX.so: undefined reference to `dlsym@GLIBC_2.34'
collect2: error: ld returned 1 exit status

I believe that I would get the same kind of problem using the host system OpenGL. I have to find one that is compatible with 2.17 ABI. I will try to use a CentOS 7 base container because I red somewhere that conda-forge uses CentOS 6 for 2.12 ABI and CentOS 7 for 2.17. I already tried but that was before I fixed the Qt problem.

Any other idea ?

By the way, if there is an OpenGL compliant with 2.17 ABI on Ubuntu, we may completely get rid of using a container.

denisri commented 1 year ago

I don't even know what is the difference between glibc version (@GLIBC_2.34 in your error messages) and glibc ABI (2.12 or 2.17). Ubuntu 22 ships glibc 2.35, and in /lib/x86_64-linux-gnu/libdl.so.2 I find symbols like GLIBC_2.2.5, GLIBC_2.3.3, GLIBC_2.3.4, with 3 numbers, none corresponding to the glibc version... OK it doesn't help, but just to say I'm trying... :)

denisri commented 1 year ago

found that:

https://gist.github.com/ernstki/593076d2d164ca51d4a6c0a15731846f

I don't know how helpful it might be, but the script there, libcversions.sh actually prints a set of supported ABIs.

sapetnioc commented 1 year ago

In the meantime, I am trying to stop using a container since OpenGL was the only dependency. I will have the same linking problem but directly with my system. I have to tune CMake and gcc so that the first one finds OpenGL but the second one does not find incompatible includes on host system...

sapetnioc commented 1 year ago

I tried with anaconda repository. There are many packages missing. I think we have no other choice than to keep using conda-forge. But I also think this may work at the end because (on a suggestion of @denisri), I tested a small OpenGL PyQt script and it works. We may just have to find the good linker options. I will retry using libs given by ldd libQt5OpenGL.so.5.15.8 as a guide.

import sys
from OpenGL.GL import *
from OpenGL.GLU import *
from PyQt5 import QtGui
from PyQt5.QtOpenGL import *
from PyQt5 import QtCore, QtWidgets, QtOpenGL

class Ui_MainWindow(QtWidgets.QWidget):
    def __init__(self, parent=None):
        super(Ui_MainWindow, self).__init__()
        self.widget = glWidget()
        self.button = QtWidgets.QPushButton('Test', self)
        mainLayout = QtWidgets.QHBoxLayout()
        mainLayout.addWidget(self.widget)
        mainLayout.addWidget(self.button)
        self.setLayout(mainLayout)

class glWidget(QGLWidget):
    def __init__(self, parent=None):
        QGLWidget.__init__(self, parent)
        self.setMinimumSize(640, 480)

    def paintGL(self):
        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
        glLoadIdentity()
        glTranslatef(-2.5, 0.5, -6.0)
        glColor3f( 1.0, 1.5, 0.0 );
        glPolygonMode(GL_FRONT, GL_FILL);
        glBegin(GL_TRIANGLES)
        glVertex3f(4.0,-2.4,0.0)
        glVertex3f(5.2,0.0,0.0)
        glVertex3f(5.8,-2.4,0.0)
        glEnd()
        glFlush()

    def initializeGL(self):
        glClearDepth(1.0)              
        glDepthFunc(GL_LESS)
        glEnable(GL_DEPTH_TEST)
        glShadeModel(GL_SMOOTH)
        glMatrixMode(GL_PROJECTION)
        glLoadIdentity()                    
        gluPerspective(45.0,1.33,0.1, 100.0) 
        glMatrixMode(GL_MODELVIEW)

if __name__ == '__main__':    
    app = QtWidgets.QApplication(sys.argv)    
    Form = QtWidgets.QMainWindow()
    ui = Ui_MainWindow(Form)    
    ui.show()    
    sys.exit(app.exec_())
sapetnioc commented 1 year ago

Yes !

image

Fortunately, ldd does right things and all I had to do is to make c++ use the same libraries. It was not possible using only paths because some libraries are both in conda and in the system and, among them, some must be taken from the system and others from conda. I ended up using an absolute path for one lib. Here are the options I added to the linker line: L/home/yann/casaconda/conda/lib -L/lib/x86_64-linux-gnu/ -lX11 -lGL -lGLdispatch -lGLX /lib/x86_64-linux-gnu/librt.so.1.

Now I have to find a not to ugly way to add this to bv_maker.

denisri commented 1 year ago

Is QtOpenGL runtime working with the host system libGL ? If yes we can consider building a local mesa libGL inside conda environment and use it just to link programs. It would be replaced with the host system one. If no, we should consider shipping such a software libGL somewhere to be used as a fallback.

denisri commented 1 year ago

Oh I missed a message (before last) which already answered my question. So I think the solution of building a local, temporary libGL would solve the problem without messing paths.

sapetnioc commented 1 year ago

Building mesa would probably work. To move forward, I used another solution faster to test for me. I tested it only on Aims and that requires a change in CMakeLists.txt. I made this pull request for Aims with a bit of explanation. We have to decide what is the best option (i.e. to accept or reject the pull request).

denisri commented 1 year ago

We can accept this PR as one possible solution. However it would need to modify the link options of every library or command which is using openGL. It is not too many places I admit, but several places are concerned and developers would need to setup the correct paths in their bv_maker.cfg for each build environment. I think I personally prefer the solution of building a mesa lib (and perhaps shipping it as a dev package?) because the build mechanisms would not be changed (and potentially messed). But developers would need to install (or build) this mesa lib anyway. So I'm not completely sure.

sapetnioc commented 1 year ago

I do not really know either which solution is the best. For now I am going step by step until everything gets compiled. Now, I have a new problem related to SIP ABI:

[ 44%] Generating sipbuild/aimsguisip/sipaimsguisippart0.cpp, sipbuild/aimsguisip/sipaimsguisippart1.cpp, sipbuild/aimsguisip/sipaimsguisippart2.cpp, sipbuild/aimsguisip/sipaimsguisippart3.cpp, sipbuild/aimsguisip/sipaimsguisippart4.cpp, sipbuild/aimsguisip/sipaimsguisippart5.cpp, sipbuild/aimsguisip/sipaimsguisippart6.cpp, sipbuild/aimsguisip/sipaimsguisippart7.cpp
These bindings will be built: aimsgui_VOID.
Generating the aimsgui_VOID bindings...
sip-build: /home/yann/casaconda/conda/lib/python3.10/site-packages/PyQt5/bindings/QtGui/QtGui.toml: 'QtGui' was built against ABI v12 but this module is being built against ABI v13
Error: sip exited with signal 1

The failing command is:

cd /home/yann/casaconda/build/build_files/aims-gpl/pyaimsgui/src/sip_gui && /home/yann/casaconda/build/bin/bv_sip4make -S /home/yann/casaconda/conda/bin/sip-build -c /home/yann/casaconda/build/build_files/aims-gpl/pyaimsgui/src/sip_gui/sipbuild/aimsguisip -j 8 --no-compile

The QtGui.toml contains sip-version = "6.7.5" and sip-abi-version = "12.11". CMakeCache.txt contains SIP_VERSION:STRING=6.7.9 and SIP_ABI:STRING=12.11. I do not know where this ABI v13 comes from...

If you want to reproduce, I just made a compilation from scratch using this recipe.

sapetnioc commented 1 year ago

Looks like our sip modules are generated using another ABI:

grep -r 13\\.0 ~/casaconda/build
/home/yann/casaconda/build/CMakeFiles/3.26.3/CompilerIdFortran/CMakeFortranCompilerId.F:       PRINT *, 'INFO:simulate_version[013.01]'
/home/yann/casaconda/build/CMakeFiles/3.26.3/CompilerIdFortran/CMakeFortranCompilerId.F:       PRINT *, 'INFO:simulate_version[013.00]'
/home/yann/casaconda/build/build_files/aims-free/pyaims/pyaimsalgo/src/sip/sipbuild/bindings/aimsalgosip/aimsalgosip.toml:sip-abi-version = "13.0"
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/bindings/aimssip/aimssip.toml:sip-abi-version = "13.0"
/home/yann/casaconda/build/build_files/sulci-nonfree/pysigraph/src/sip_svm/sipbuild/bindings/sigraphsvmsip/sigraphsvmsip.toml:sip-abi-version = "13.0"
/home/yann/casaconda/build/build_files/sulci-nonfree/pysigraph/src/sip/sipbuild/bindings/sigraphsip/sigraphsip.toml:sip-abi-version = "13.0"
/home/yann/casaconda/build/build_files/constellation-nonfree/src/sip/sipbuild/bindings/constelsip/constelsip.toml:sip-abi-version = "13.0"
sapetnioc commented 1 year ago

It seems that the ABI version is not passed to sip: https://github.com/brainvisa/brainvisa-cmake/blob/e4526f8d47cf5e5c68710c38358ed758a9a24fc5/cmake/modules/FindSIP.cmake#L108

denisri commented 1 year ago

Oh. Crap. This is exactly the problem I ran into using pip and PyQt5, and lead me to give up... The code you highlight just above is very recent (part of brainvisa/brainvisa-cmake@7b36ea37102d7d02a076ca7ef9ec8e94c66f4072), dating 3 days ago, and was my attempt to deal with this problem. If I understand correctly (I'm not 100% sure), sip 6.7.9 which is installed via pip or conda produces (by default, at least) ABI 13. However PyQt5 was built using ABI 12.11, which is not the defaut one for this version of sip. The sip-module command has an option --abi-version which can be used to produce a sip.h header with the desired ABI version, however the sip-build command (upper level, and this is the one which actually generates the C++ source code for bindings) does not have this option. The documentation mentions the abi-version option in the project file used by sip-build (https://www.riverbankcomputing.com/static/Docs/sip/pyproject_toml.html#tool-sip-project-section) but it apparently does not change the result, as it is a "minimum ABI version" rather than a way to force back-compatibility (this is why I have commented the code out). So I don't know how they have built PyQt5 using ABI 12.11. I thought PyQt might have been compiled using an older version of sip, and discovered that a sip5 module also exists in pip (not documented in riverbankcomputing site which only documents sip4 (as obsolete) and sip6) and also made adaptations to use it, but this sip5 uses still another ABI version (12.8 if I remember, and not 12.11) so it didn't work also. I have not tried earlier versions of sip6, but as you mention (The QtGui.toml contains sip-version = "6.7.5" and sip-abi-version = "12.11") it has really been built using sip 6.7.5 and not a older one. So there must be a way to really force the ABI but I don't know how. To explain a little further, sip6 sip-build command is using as input a "project file", pyproject.toml, and generates as outputs other .toml files (for us, for instance aims.toml, or for PyQt5, this QtGui.toml) and the C++ source code with it. So the abi-version = "13.0" in QtGui.toml or aims.toml is automatically generated by sip-build, even if we have specified 12.11 in the main project input file. I did not find another way but there must be one since PyQt5 has been built this way.

sapetnioc commented 1 year ago

I had a look at PyQt code and I believe, it tries to keep the major version even if it can take a more recent minor version. So I removed comment on the sip project toml template and it had an effect. The good news is that it is taken into account. The bad news is that I get errors:

[ 44%] Generating sipbuild/aimssip/sipaimssippart0.cpp, sipbuild/aimssip/sipaimssippart1.cpp, sipbuild/aimssip/sipaimssippart2.cpp, sipbuild/aimssip/sipaimssippart3.cpp, sipbuild/aimssip/sipaimssippart4.cpp, sipbuild/aimssip/sipaimssippart5.cpp, sipbuild/aimssip/sipaimssippart6.cpp, sipbuild/aimssip/sipaimssippart7.cpp, sipbuild/aimssip/sipaimssippart8.cpp, sipbuild/aimssip/sipaimssippart9.cpp, sipbuild/aimssip/sipaimssippart10.cpp, sipbuild/aimssip/sipaimssippart11.cpp, sipbuild/aimssip/sipaimssippart12.cpp, sipbuild/aimssip/sipaimssippart13.cpp, sipbuild/aimssip/sipaimssippart14.cpp, sipbuild/aimssip/sipaimssippart15.cpp
These bindings will be built: aims.
Generating the aims bindings...
sip-build: /home/yann/casaconda/build/share/aims-5.2/sip/rcobject.sip: line 253: column 3: 'carto::RCObject::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/genericobject.sip: line 106: column 3: 'carto::GenericObject::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_GenericObject.sip: line 169: column 3: 'rc_ptr_GenericObject::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Tree.sip: line 169: column 3: 'rc_ptr_Tree::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Graph.sip: line 166: column 3: 'rc_ptr_Graph::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Vertex.sip: line 166: column 3: 'rc_ptr_Vertex::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Edge.sip: line 166: column 3: 'rc_ptr_Edge::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_DataSource.sip: line 178: column 3: 'rc_ptr_DataSource::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_DataSourceInfo.sip: line 178: column 3: 'rc_ptr_DataSourceInfo::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_2_VOID.sip: line 166: column 3: 'rc_ptr_AimsTimeSurface_2_VOID::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_3_VOID.sip: line 178: column 3: 'rc_ptr_AimsTimeSurface_3_VOID::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_4_VOID.sip: line 166: column 3: 'rc_ptr_AimsTimeSurface_4_VOID::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_2_FLOAT.sip: line 166: column 3: 'rc_ptr_AimsTimeSurface_2_FLOAT::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_3_FLOAT.sip: line 166: column 3: 'rc_ptr_AimsTimeSurface_3_FLOAT::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_4_FLOAT.sip: line 166: column 3: 'rc_ptr_AimsTimeSurface_4_FLOAT::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_2_POINT2DF.sip: line 166: column 3: 'rc_ptr_AimsTimeSurface_2_POINT2DF::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_3_POINT2DF.sip: line 166: column 3: 'rc_ptr_AimsTimeSurface_3_POINT2DF::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AimsTimeSurface_4_POINT2DF.sip: line 166: column 3: 'rc_ptr_AimsTimeSurface_4_POINT2DF::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_TimeTexture_U8.sip: line 169: column 3: 'rc_ptr_TimeTexture_U8::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_TimeTexture_S16.sip: line 169: column 3: 'rc_ptr_TimeTexture_S16::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_TimeTexture_U16.sip: line 169: column 3: 'rc_ptr_TimeTexture_U16::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_TimeTexture_S32.sip: line 169: column 3: 'rc_ptr_TimeTexture_S32::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_TimeTexture_U32.sip: line 169: column 3: 'rc_ptr_TimeTexture_U32::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_TimeTexture_FLOAT.sip: line 169: column 3: 'rc_ptr_TimeTexture_FLOAT::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_TimeTexture_DOUBLE.sip: line 166: column 3: 'rc_ptr_TimeTexture_DOUBLE::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_TimeTexture_POINT2DF.sip: line 170: column 3: 'rc_ptr_TimeTexture_POINT2DF::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_BucketMap_VOID.sip: line 170: column 3: 'rc_ptr_BucketMap_VOID::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_BucketMap_S16.sip: line 167: column 3: 'rc_ptr_BucketMap_S16::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_BucketMap_U16.sip: line 167: column 3: 'rc_ptr_BucketMap_U16::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_BucketMap_S32.sip: line 167: column 3: 'rc_ptr_BucketMap_S32::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_BucketMap_U32.sip: line 167: column 3: 'rc_ptr_BucketMap_U32::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_BucketMap_FLOAT.sip: line 167: column 3: 'rc_ptr_BucketMap_FLOAT::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_BucketMap_DOUBLE.sip: line 167: column 3: 'rc_ptr_BucketMap_DOUBLE::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Transformation3d.sip: line 178: column 3: 'rc_ptr_Transformation3d::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Transformation.sip: line 178: column 3: 'rc_ptr_Transformation::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_AffineTransformation3d.sip: line 178: column 3: 'rc_ptr_AffineTransformation3d::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Site.sip: line 178: column 3: 'rc_ptr_Site::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_SiteIterator.sip: line 178: column 3: 'rc_ptr_SiteIterator::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Interpolator.sip: line 178: column 3: 'rc_ptr_Interpolator::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_SparseMatrix.sip: line 178: column 3: 'rc_ptr_SparseMatrix::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_SparseOrDenseMatrix.sip: line 178: column 3: 'rc_ptr_SparseOrDenseMatrix::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_BOOL.sip: line 169: column 3: 'rc_ptr_Volume_BOOL::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_U8.sip: line 169: column 3: 'rc_ptr_Volume_U8::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_S16.sip: line 169: column 3: 'rc_ptr_Volume_S16::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_U16.sip: line 169: column 3: 'rc_ptr_Volume_U16::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_S32.sip: line 169: column 3: 'rc_ptr_Volume_S32::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_U32.sip: line 169: column 3: 'rc_ptr_Volume_U32::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_FLOAT.sip: line 169: column 3: 'rc_ptr_Volume_FLOAT::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_DOUBLE.sip: line 169: column 3: 'rc_ptr_Volume_DOUBLE::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_RGB.sip: line 170: column 3: 'rc_ptr_Volume_RGB::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_RGBA.sip: line 170: column 3: 'rc_ptr_Volume_RGBA::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_HSV.sip: line 167: column 3: 'rc_ptr_Volume_HSV::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_CFLOAT.sip: line 167: column 3: 'rc_ptr_Volume_CFLOAT::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_CDOUBLE.sip: line 167: column 3: 'rc_ptr_Volume_CDOUBLE::__hash__' must return a long
/home/yann/casaconda/build/share/aims-5.2/sip/rcptr_Volume_POINT3DF.sip: line 167: column 3: 'rc_ptr_Volume_POINT3DF::__hash__' must return a long
Error: sip exited with signal 1
sapetnioc commented 1 year ago

I think the problem may be due to to the test here. I will try to remove it. If it works, I do not know if it is possible to make this test on the ABI version instead of sip version.

sapetnioc commented 1 year ago

If I remove the command, I get:

/home/yann/casaconda/build/share/aims-5.2/sip/system_VOID.sip: line 20: column 14: 'Py_hash_t' is unexpected

Is there a way to replace this typedef definition in sip 6 ?

denisri commented 1 year ago

It's probably related to that, yes. SIP_VERSION is passed to maketemplates.py by the build infrastructure, but it doesn't pass the ABI version. Perhaps we can change it to also pass it, and the test in system.tpl would rather be based on the ABI version than on SIP version. Py_hash_t and Py_ssize_t are builtin in new sip or sip ABI versions, and were not defined earlier, hence the typedefs for earlier versions. I didn't understand if you define the typedefs using ABI 12 or not... ?

sapetnioc commented 1 year ago

Ok. Finally I reverted on the test but replaced all Py_hash_t by long in sip files. I really don't know if it is the right way to go. But it allows the building to move forward just to fail a little bit later:

/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:53785:1: error: too many initializers for 'sipTypeDef' {aka '_sipTypeDef'}
53785 |     &sipTypeDef_aimssip_vector_DOUBLE.ctd_base,
      | ^
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:53788:8: error: 'sipEnumMemberDef' does not name a type; did you mean 'sipEnumTypeDef'?
53788 |     &sipTypeDef_aimssip_vector_FLOAT_iterator.ctd_base,
      |        ^~~~~~~~~~~~~~~~
      |        sipEnumTypeDef
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:54852:5: error: 'enummembers' was not declared in this scope
54852 |     sipNoMethod(sipParseErr, sipName_Referential, sipName_setOrientation, SIP_NULLPTR);
      |     ^~~~~~~~~~~
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:56426:8: error: 'sipEnumMemberDef' does not name a type; did you mean 'sipEnumTypeDef'?
56426 |             sipCpp = new  ::LightResampler_CDOUBLE();
      |        ^    ~~~~~~~~~~~
      |        sipEnumTypeDef
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:56493:12: error: 'enummembers_carto_Referential' was not declared in this scope; did you mean 'methods_carto_Referential'?
56493 | "flipping operations.");
      |            ^~~~~~~~~~~~~                
      |            methods_carto_Referential
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:56521:1: error: too many initializers for 'sipTypeDef' {aka '_sipTypeDef'}
56521 |     SIP_NULLPTR,
      | ^
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:57039:1: error: too many initializers for 'sipTypeDef' {aka '_sipTypeDef'}
57039 | #include <pyaims/object/numconv.h>
      | ^
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:57530:1: error: too many initializers for 'sipTypeDef' {aka '_sipTypeDef'}
57530 | #line 58538 "/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp"
      | ^
/home/yann/casaconda/build/build_files/aims-free/pyaims/src/sip/sipbuild/aimssip/sipaimssippart0.cpp:58021:1: error: too many initializers for 'sipTypeDef' {aka '_sipTypeDef'}
58021 | #ifndef PYAIMSSIP_VECTOR_S32_DEFINED
      | ^
sapetnioc commented 1 year ago

I edited my previous post, I wrote Py_ssize_t instead of Py_hash_t.

denisri commented 1 year ago

I don't remember exactly but I think the typedef was here because depending on sip/abi/python versions it was not defined as the same type: long may not work with all sip versions. You can try like this, but I think we would need to find another solution later. Why does the typedef thing not work ?

The errors you get seem to mean (to me) that there is a discrepency between the sip ABI / sip.h file, and the generated code. In FindSIP.cmake, did you also uncomment the lines:

            ## not taken into account by sip-build, so it is useless and breaks
            # set( _added_flags "--abi-version" "${SIP_ABI}" )

If not, you should ;)

sapetnioc commented 1 year ago

It is working !

I now have to fix OpenGL for libanatomist. By the way, I noticed that the generation of sip files is now slow and always done twice when running make. We will probably have to look at his old known problem.

denisri commented 1 year ago

Wondeful !! So there will be a conda package for pyanatomist ?? ;) 8)

sapetnioc commented 1 year ago

If it goes to the end of the build (it is a bit long because everything is compiles each time I make a change in any CMakeLists.txt) it will be possible to package all our software. But I let others handle the Windows and Mac platforms...

sapetnioc commented 1 year ago

image We are getting close to the end !

For OpenGL compilation, I had to do two things:

I still have a sip compilation problem in Axon:

[ 61%] Generating sipbuild/somaqt/sipsomaqtpart0.cpp, sipbuild/somaqt/sipsomaqtpart1.cpp, sipbuild/somaqt/sipsomaqtpart2.cpp, sipbuild/somaqt/sipsomaqtpart3.cpp, sipbuild/somaqt/sipsomaqtpart4.cpp, sipbuild/somaqt/sipsomaqtpart5.cpp, sipbuild/somaqt/sipsomaqtpart6.cpp, sipbuild/somaqt/sipsomaqtpart7.cpp
cd /home/yann/casaconda/build/build_files/axon/src/sipsomaqt && /home/yann/casaconda/build/bin/bv_sip4make -S /home/yann/casaconda/conda/bin/sip-build -c /home/yann/casaconda/build/build_files/axon/src/sipsomaqt/sipbuild/somaqt -j 8 --no-compile
These bindings will be built: soma_qt6.
Generating the soma_qt6 bindings...
sip-build: /home/yann/casaconda/conda/lib/python3.10/site-packages/PyQt5/bindings/QtCore/QtCoremod.sip: line 27: column 1: 'WS_X11' has already been defined as a qualifier
/home/yann/casaconda/conda/lib/python3.10/site-packages/PyQt5/bindings/QtCore/QtCoremod.sip: line 27: column 1: 'WS_WIN' has already been defined as a qualifier
Error: sip exited with signal 1
make[2]: *** [build_files/axon/src/sipsomaqt/CMakeFiles/somaqt.dir/build.make:81 : build_files/axon/src/sipsomaqt/sipbuild/somaqt/sipsomaqtpart0.cpp] Erreur 1

This is probably due to the use of soma_qt6.sip. It looks that this test uses the wrong variable (sip instead of PyQt).