search

dev-cafe / autocmake

CMake plugin composer.
http://autocmake.org
BSD 3-Clause "New" or "Revised" License
41 stars 18 forks source link

setup swallows stdout screen messages when stderr message appear #174

Closed miroi closed 7 years ago

miroi commented 8 years ago

Simple modification of autocmake/test/fc/src/CMakeLists.txt

add_executable(example example.f90 module.f90)
if (FOO)
 message(STATUS "FOO active!")
 message(WARNING "FOO warning")
endif()

gives i) nothing swallowed when FOO inactive:

milias@lxir073.gsi.de:~/Work/qch/software/small_projects/autocmake/test/fc/.python ./setup --cmake-options="-D FOO=OFF" build_x1
FC=gfortran cmake -DEXTRA_FCFLAGS="''" -DCMAKE_BUILD_TYPE=release -G "Unix Makefiles" -D FOO=OFF /u/milias/Work/qch/software/small_projects/autocmake/test/fc

-- The C compiler identification is GNU 4.7.2
-- The CXX compiler identification is GNU 4.7.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is GNU 4.7.2
-- Check for working Fortran compiler: /usr/bin/gfortran
-- Check for working Fortran compiler: /usr/bin/gfortran  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 -- yes
-- Configuring done
-- Generating done
-- Build files have been written to: /u/milias/Work/qch/software/small_projects/autocmake/test/fc/build_x1

   configure step is done
   now you need to compile the sources:
   $ cd build_x1
   $ make

or all STATUS messages are swallowed when FOO variable is active:

milias@lxir073.gsi.de:~/Work/qch/software/small_projects/autocmake/test/fc/.python ./setup --cmake-options="-D FOO=ON" build_x
FC=gfortran cmake -DEXTRA_FCFLAGS="''" -DCMAKE_BUILD_TYPE=release -G "Unix Makefiles" -D FOO=ON /u/milias/Work/qch/software/small_projects/autocmake/test/fc

CMake Warning at src/CMakeLists.txt:4 (message):
  FOO warning
miroi commented 8 years ago

Hi Rado,

it is interesting that this behavior above is only for the screen (terminal); the CDash web, however, shows both STATUS and WARNING messages:

https://testboard.org/cdash/buildSummary.php?buildid=82864

miroi commented 7 years ago

Hi Rado,

the CMAke warning message swallows following stdout message in current Dirac, see below. Is there a way to fix it with fresh autocmake ?

-- C compiler flags:  -xHost -g -wd981 -wd279 -wd383 -wd1572 -wd177 
-- CXX compiler: /home/f113112/work/qch/software/openmpi-2.1.1_intel-15/bin/mpiCC
-- CXX compiler version: Intel 15.0.2
-- CXX compiler flags:  -xHost -Wno-unknown-pragmas 
-- Static linking: False
-- 64-bit integers: True
-- MPI parallelization: True
-- MPI launcher: /home/f113112/work/qch/software/openmpi-2.1.1_intel-15/bin/mpiexec
-- Explicit libraries: -lopenblas
-- Compile definitions: HAVE_MPI;VAR_MPI;VAR_MPI2;VAR_MPI_32BIT_INT;MOD_UNRELEASED;SYS_LINUX;PRG_DIRAC;INT_STAR8;INSTALL_WRKMEM=64000000;HAS_PCMSOLVER;BUILD_GEN1INT;HAS_PELIB;MOD_QCORR;HAS_STIELTJES;MOD_INTEREST;MOD_LAO_REARRANGED;MOD_MCSCF_spinfree;MOD_AOOSOC;MOD_ESR;MOD_KRCC;MOD_SRDFT
-- Could NOT find Sphinx (missing:  SPHINX_EXECUTABLE) 
-- Adding target release
-- Set CDash timeout for every single test set to 1500 seconds. Owerwritten by pam timeout setting.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/f113112/work/qch/software/dirac/production_trunk/build_openmpi_intel15_i8_xhost_openblas

CMake Warning:
  Manually-specified variables were not used by the project:

    MATH_LIB_SEARCH_ORDER

f113112@peregrine.hpc.rug.nl:~/work/qch/software/dirac/production_trunk/.cd build_openmpi_intel15_i8_xhost_openblas
miroi commented 7 years ago

Hi Rado,

returning to this issue (https://gitlab.com/dirac/dirac/issues/246 ) - setup swallows stdout screen messages when stderr message appear, in DIRAC:

milias@login.grid.umb.sk:~/Work/qch/software/dirac/production_trunk/../setup --fc=pgf90 --cc=pgcc --cxx=g++ --int64 --cmake-options="-D ENABLE_STIELTJES=OFF -D ENABLE_PCMSOLVER=OFF -D ENABLE_GEN1INT=OFF -DENABLE_PELIB=OFF"  build_nopelib_nogen1_nostiel_nopcm_pgi_g++_mkl_i8                           
FC=pgf90 CC=pgcc CXX=g++ cmake -DEXTRA_FCFLAGS="''" -DEXTRA_CFLAGS="''" -DEXTRA_CXXFLAGS="''" -DPREPROCESSOR_DEFINITIONS="''" -DPYTHON_INTERPRETER="''" -DENABLE_BLAS=auto -DENABLE_LAPACK=auto -DMKL_FLAG=off -DMATH_LIB_SEARCH_ORDER="MKL;ESSL;OPENBLAS;ATLAS;ACML;SYSTEM_NATIVE" -DBLAS_LANG=Fortran -DLAPACK_LANG=Fortran -DENABLE_MPI=False -DENABLE_CODE_COVERAGE=False -DENABLE_STATIC_LINKING=False -DENABLE_PROFILING=False -DENABLE_RUNTIMECHECK=False -DENABLE_64BIT_INTEGERS=True -DEXPLICIT_LIBS="off" -DENABLE_PCMSOLVER=ON -DPCMSOLVER_ROOT='' -DCMAKE_BUILD_TYPE=release -G "Unix Makefiles" -D ENABLE_STIELTJES=OFF -D ENABLE_PCMSOLVER=OFF -D ENABLE_GEN1INT=OFF -DENABLE_PELIB=OFF /home/milias/Work/qch/software/dirac/production_trunk

-- The Fortran compiler identification is PGI 1.1
-- The C compiler identification is PGI 13.10.0
-- The CXX compiler identification is GNU 4.4.7
-- Check for working Fortran compiler: /mnt/apps/pgi/linux86-64/13.10/bin/pgf90
-- Check for working Fortran compiler: /mnt/apps/pgi/linux86-64/13.10/bin/pgf90  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /mnt/apps/pgi/linux86-64/13.10/bin/pgf90 supports Fortran 90
-- Checking whether /mnt/apps/pgi/linux86-64/13.10/bin/pgf90 supports Fortran 90 -- yes
-- Check for working C compiler: /mnt/apps/pgi/linux86-64/13.10/bin/pgcc
-- Check for working C compiler: /mnt/apps/pgi/linux86-64/13.10/bin/pgcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /mnt/apps/python27/bin/python (found version "2.7.12") 
-- BLAS will be searched for based on MKLROOT=/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl
-- Searching for BLAS using search order MKL;ESSL;OPENBLAS;ATLAS;ACML;SYSTEM_NATIVE
-- Setting -DHAVE_MKL_BLAS
-- Found BLAS: MKL (-Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_ilp64.so;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_pgi_thread.so;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_core.so;/usr/lib64/libpthread.so;/usr/lib64/libm.so;-mp;-Wl,--end-group)
-- LAPACK will be searched for based on MKLROOT=/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl
-- Searching for LAPACK using search order MKL;ESSL;OPENBLAS;ATLAS;ACML;SYSTEM_NATIVE
-- Setting -DHAVE_MKL_LAPACK
-- Found LAPACK: MKL (-Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_lapack95_ilp64.a;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_ilp64.so;-mp;-Wl,--end-group)
-- MATH_LIB_SEARCH_ORDER set to MKL;ESSL;OPENBLAS;ATLAS;ACML;SYSTEM_NATIVE
-- Enable profiling: False
-- Enable run-time checking: False
-- Configure-time Linked math libs integer compatibility test ENABLED
-- Performing compile and run test of blas integer size (can deactivate with -D ENABLE_MATH_INT_TEST=OFF) 
-- Performing Test blas_test_compiled
-- Performing Test blas_test_compiled - Success
-- Performing Test blas_test_run
-- Performing Test blas_test_run - Failed
-- Test of blas integer size SUCCESS .. blas is integer*8 
-- We have BLAS-integer*8
-- Set CDash timeout for every single test set to 1500 seconds. Owerwritten by pam timeout setting.
-- Polarizable Continuum Model via PCMSolver DISABLED
-- Gen1Int module: OFF
-- PElib module: OFF
-- Found Git: /usr/bin/git (found version "1.7.1") 
-- The XCFun submodule ENABLED
-- Davidson-type +Q corrections for (MR)CISD: ON
-- ESR property module: ON
-- Stieltjes external module DISABLED
-- Interest library: ON
-- KRCC module: ON
-- Enable compilation of standalone relccsd.x: ON
-- OpenRSP library: OFF
-- LAO properties without connection matrices: ON
-- Spinfree MCSCF module: ON
-- Atomic oo-order spin-orbit correction module: ON
-- srDFT module: ON
-- Specialized tutorial tests DISABLED
-- Unit control tests DISABLED
-- Versions from CMake variable (1.1) and Python script (13.10) for Fortran compiler are not equal!
-- Not supported version of Fortran compiler according to CMake. Found: 1.1, Required: 12.5
-- User name: milias
-- Host name: login
-- Operating system: Linux-2.6.32-642.15.1.el6.x86_64
-- CMake version: 3.3.0
-- CMake generator: Unix Makefiles
-- CMake build type: release
-- Configuration time: 2017-07-08 16:17:56.098175
-- Python version: 2.7.1
-- Fortran compiler: /mnt/apps/pgi/linux86-64/13.10/bin/pgf90
-- Fortran compiler version: PGI 13.10
-- Fortran compiler flags:   -Mpreprocess -Kieee -DVAR_PGF90 -traceback -g -i8
-- C compiler: /mnt/apps/pgi/linux86-64/13.10/bin/pgcc
-- C compiler version: PGI 13.10
-- C compiler flags:  -g -traceback
-- CXX compiler: /usr/bin/g++
-- CXX compiler version: GNU 4.4.7
-- CXX compiler flags:  -g -Wall -Wno-unknown-pragmas -Wno-sign-compare -Woverloaded-virtual -Wwrite-strings -Wno-unused
-- Static linking: False
-- 64-bit integers: True
-- MPI parallelization: False
-- Math libraries: -Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_lapack95_ilp64.a;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_ilp64.so;-mp;-Wl,--end-group;-Wl,--start-group;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_ilp64.so;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_pgi_thread.so;/mnt/apps/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_core.so;/usr/lib64/libpthread.so;/usr/lib64/libm.so;-mp;-Wl,--end-group
-- Compile definitions: HAVE_MKL_BLAS;HAVE_MKL_LAPACK;MOD_UNRELEASED;SYS_LINUX;PRG_DIRAC;INT_STAR8;INSTALL_WRKMEM=64000000;BUILD_GEN1INT;MOD_QCORR;MOD_INTEREST;MOD_LAO_REARRANGED;MOD_MCSCF_spinfree;MOD_AOOSOC;MOD_ESR;MOD_KRCC;MOD_SRDFT
-- Found Sphinx: /usr/bin/sphinx-build  
-- Adding target release
-- Set CDash timeout for every single test set to 1500 seconds. Owerwritten by pam timeout setting.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/milias/Work/qch/software/dirac/production_trunk/build_nopelib_nogen1_nostiel_nopcm_pgi_g++_mkl_i8

CMake Warning:
  Manually-specified variables were not used by the project:

    PCMSOLVER_ROOT

Could you reopen this issue please ?

bast commented 7 years ago

OK so what should happen here instead? What messages are swallowed?

miroi commented 7 years ago

This CMake message is missing (is swallowed):

configure step is done
   now you need to compile the sources:
   $ cd build_nopelib_nogen1_nostiel_nopcm_pgi_g++_mkl_i8
   $ make
bast commented 7 years ago

The problem is that CMake warnings go to stderr and we decide to stop on stderr: https://github.com/coderefinery/autocmake/blob/master/autocmake/configure.py#L100-L102

So we either continue or we print more clearly that the configuration has stopped with a message to stderr. I agree that currently it is weird, since it just stops out of the blue.

miroi commented 7 years ago

Stopping the configuration for mere warning message isn't fair. I suggest not to stop configuration and continue with a stdout message about user's risk. Like we detected stderr....decide if you compile etc

bast commented 7 years ago

But how do we then know it is not a real error which also goes to stderr? IMO the weird behavior is CMake sending warnings to stderr.

miroi commented 7 years ago

Python lower level routines, if hurt, do stop at error. Cmake also stops at error at its own. Why to introduce own stderr catching, if other involved parties (python, cmake) do care as well ?

bast commented 7 years ago

OK we cannot just continue (in case CMake catches an error) but we need better mechanism to detect whether the CMake step worked and it seems that just listening to the stderr stream is not good enough since it is too sensitive. Suggestions welcome. I agree that currently it is annoying.

miroi commented 7 years ago

If stderr stream caught, let us flush message: " stderr detected...this could or might not autocmake configuration process. Continuation is at your own risk."

It would be too difficult to analyse all possible stderrs.

bast commented 7 years ago

If stderr is non-empty, we should print it.

How about we check whether stdout contains:

-- Configuring done
-- Generating done
-- Build files have been written to: /path/foo/build