search

statgen / Minimac4

GNU General Public License v3.0
56 stars 20 forks source link

CentOS 6 install failed #14

Open yutoubb opened 6 years ago

yutoubb commented 6 years ago

Scanning dependencies of target minimac4 [ 8%] Building CXX object CMakeFiles/minimac4.dir/src/Analysis.cpp.o [ 16%] Building CXX object CMakeFiles/minimac4.dir/src/AnalysisChunks.cpp.o [ 25%] Building CXX object CMakeFiles/minimac4.dir/src/DosageData.cpp.o [ 33%] Building CXX object CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o [ 41%] Building CXX object CMakeFiles/minimac4.dir/src/Imputation.cpp.o [ 50%] Building CXX object CMakeFiles/minimac4.dir/src/ImputationStatistics.cpp.o [ 58%] Building CXX object CMakeFiles/minimac4.dir/src/Main.cpp.o [ 66%] Building CXX object CMakeFiles/minimac4.dir/src/MarkovModel.cpp.o [ 75%] Building CXX object CMakeFiles/minimac4.dir/src/MarkovParameters.cpp.o [ 83%] Building CXX object CMakeFiles/minimac4.dir/src/Unique.cpp.o [ 91%] Building CXX object CMakeFiles/minimac4.dir/src/Estimation.cpp.o [100%] Linking CXX executable minimac4 CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::CheckGeneticMapFile()': Analysis.cpp:(.text+0x1abe): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::readm3vcfFileChunk(int, HaplotypeSet&)': Analysis.cpp:(.text+0x2871): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::InitializeRefFileStream(String&)': Analysis.cpp:(.text+0x32ff): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' Analysis.cpp:(.text+0x3340): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' Analysis.cpp:(.text+0x378b): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o:Analysis.cpp:(.text+0x40b0): more undefined references to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' follow CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In functionHaplotypeSet::GetVariantInfofromVCFFile(String&, String, AllVariable&)': HaplotypeSet.cpp:(.text+0xad0f): undefined reference to VcfRecordFilter::getString(int)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In functionHaplotypeSet::readm3vcfFile(String, String, int, int, int)': HaplotypeSet.cpp:(.text+0xc406): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' HaplotypeSet.cpp:(.text+0xc52c): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' HaplotypeSet.cpp:(.text+0xcfd2): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' HaplotypeSet.cpp:(.text+0xdb78): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Estimation.cpp.o: In function Estimation::readm3vcfFileChunk(int, HaplotypeSet&)': Estimation.cpp:(.text+0x1de9): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Estimation.cpp.o:Estimation.cpp:(.text+0x5bcd): more undefined references to `InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' follow collect2: ld returned 1 exit status make[2]: [minimac4] Error 1 make[1]: [CMakeFiles/minimac4.dir/all] Error 2 make: *** [all] Error 2 Binary created at /release-build/minimac4

yutoubb commented 6 years ago

I've been struggling to install Minimac4 for a couple of days. After installed python virtualenv for cget and changed cmake and other shared libs, I got above error in make step.

By the way, would it be possible to provide linux executable files like Minimac3? For users without root privileges and using an old cluster like me, it's very difficult to get every thing right for compiling.

jonathonl commented 6 years ago

Can you please provide the OS version you are running?

The problem is that libStatGen cannot be found for linking. I'm assuming you ran the install.sh script and there are error messages in that script's log files. I would recommend not using that script. Instead you can run the following from the Minimac4 source directory:

cget install -f ./requirements.txt
rm -rf build && mkdir build && cd build
cmake -DCMAKE_TOOLCHAIN_FILE=../cget/cget/cget.cmake .. 
make

Alternatively, there are binary packages in the releases section. There are system dependent ways to extract into a non-root directory. For Debian it will look something like dpkg -x pkg.deb dest_dir.

Side note: you do not need a virtual environment to install cget without root. Instead run pip install --user cget.

yutoubb commented 6 years ago

Thank you for your reply! The OS should be a customized version of Centos 6. Our system default python is 2.6, so I used a python 2.7 which installed by other user. I have tried to install the libStatGen in another folder (by git clone & make all if I remember correctly) and got no error. And when I ran the install.sh I have checked the install.log but it seems fine.

Anyway, I find the .rpm in the release folder, and use follow command to get the executable file. rpm2cpio ctags-5.8-2.el6.x86_64.rpm |cpio -idvm and then use glibc 2.17 to run it.

yutoubb commented 5 years ago

$ cget install -f ./requirements.txt Downloading https://github.com/statgen/libStatGen/archive/HEAD.tar.gz [######################################################################] 100% Extracting archive /some_path/Minimac4/cget/cget/build/tmp-62ee1d7ab3794245974f551b30cb3092/HEAD.tar.gz ... -- The C compiler identification is GNU 4.4.7 -- The CXX compiler identification is GNU 4.4.7 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Configuring done -- Generating done CMake Warning: Manually-specified variables were not used by the project:

BUILD_TESTING
CGET_CMAKE_DIR
CGET_CMAKE_ORIGINAL_SOURCE_FILE

-- Build files have been written to: /some_path/Minimac4/cget/cget/build/tmp-62ee1d7ab3794245974f551b30cb3092/build Scanning dependencies of target libStatGen [100%] Builing libStatGen ... make[3]: warning: jobserver unavailable: using -j1. Add +' to parent make rule. [100%] Built target libStatGen [100%] Builing libStatGen ... make[3]: warning: jobserver unavailable: using -j1. Add+' to parent make rule. [100%] Built target libStatGen Install the project... -- Install configuration: "Release" -- Installing: /some_path/Minimac4/cget/cget/pkg/statgen__libStatGen/install/include/SamHeaderTag.h -- Installing: /some_path/Minimac4/Minimac4/cget/cget/pkg/statgen__libStatGen/install/include/SamRecordPool.h

...too many lines..........

-- Installing: /some_path/Minimac4/cget/cget/pkg/statgen__libStatGen/install/include/VcfGenotypeField.h -- Installing: /some_path/Minimac4/cget/cget/pkg/statgen__libStatGen/install/lib/libStatGen.a Successfully installed statgen/libStatGen

$ rm -rf build && mkdir build && cd build $ cmake -DCMAKE_TOOLCHAIN_FILE=../cget/cget/cget.cmake .. -- The C compiler identification is GNU 4.4.7 -- The CXX compiler identification is GNU 4.4.7 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Try OpenMP C flag = [-fopenmp] -- Performing Test OpenMP_FLAG_DETECTED -- Performing Test OpenMP_FLAG_DETECTED - Success -- Try OpenMP CXX flag = [-fopenmp] -- Performing Test OpenMP_FLAG_DETECTED -- Performing Test OpenMP_FLAG_DETECTED - Success -- Found OpenMP: -fopenmp -- Found ZLIB: /zfssz5/BC_PS/huangshujia/biosoft/local/lib/libz.so (found version "1.2.8") -- Configuring done -- Generating done -- Build files have been written to: /some_path/Minimac4/build

(mini) zhouyuwen@software-install:/some_path/Minimac4/build$ make Scanning dependencies of target minimac4 [ 8%] Building CXX object CMakeFiles/minimac4.dir/src/Analysis.cpp.o [ 16%] Building CXX object CMakeFiles/minimac4.dir/src/AnalysisChunks.cpp.o [ 25%] Building CXX object CMakeFiles/minimac4.dir/src/DosageData.cpp.o [ 33%] Building CXX object CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o [ 41%] Building CXX object CMakeFiles/minimac4.dir/src/Imputation.cpp.o [ 50%] Building CXX object CMakeFiles/minimac4.dir/src/ImputationStatistics.cpp.o [ 58%] Building CXX object CMakeFiles/minimac4.dir/src/Main.cpp.o [ 66%] Building CXX object CMakeFiles/minimac4.dir/src/MarkovModel.cpp.o [ 75%] Building CXX object CMakeFiles/minimac4.dir/src/MarkovParameters.cpp.o [ 83%] Building CXX object CMakeFiles/minimac4.dir/src/Unique.cpp.o [ 91%] Building CXX object CMakeFiles/minimac4.dir/src/Estimation.cpp.o [100%] Linking CXX executable minimac4 CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::AppendtoMainLooVcfFaster(int, int)': Analysis.cpp:(.text+0x210a): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::AppendtoMainVcfFaster(int, int)': Analysis.cpp:(.text+0x31c1): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' Analysis.cpp:(.text+0x3586): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In functionAnalysis::readm3vcfFileChunk(int, HaplotypeSet&)': Analysis.cpp:(.text+0x3d96): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In functionAnalysis::InitializeRefFileStream(String&)': Analysis.cpp:(.text+0x7cd0): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o:Analysis.cpp:(.text+0x7cff): more undefined references toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' follow CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::GetVariantInfofromVCFFile(String&, String, AllVariable&)': HaplotypeSet.cpp:(.text+0x43a2): undefined reference toVcfRecordFilter::getString(int)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::GetSummary(InputFile*)': HaplotypeSet.cpp:(.text+0x6417): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::ReadM3VCFChunkingInformation(String&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)': HaplotypeSet.cpp:(.text+0x6d9d): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::DetectFileType(String)': HaplotypeSet.cpp:(.text+0x8b64): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::readm3vcfFile(String, String, int, int, int)': HaplotypeSet.cpp:(.text+0x92bb): undefined reference toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' HaplotypeSet.cpp:(.text+0x93e2): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o:HaplotypeSet.cpp:(.text+0x9b8e): more undefined references toInputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' follow collect2: ld returned 1 exit status make[2]: [minimac4] Error 1 make[1]: [CMakeFiles/minimac4.dir/all] Error 2 make: *** [all] Error 2

yutoubb commented 5 years ago

Actually I did run each steps separately before, and just repeated the process again and still get error at the last make step.

jonathonl commented 5 years ago

I spun up a CentOS 6 VM and figured out what's going. There's a silent error due to a bug in gcc v4.4 when building libStatGen.a and the archive is left incomplete. I made a fix to the libStatGen repo. Remove the cget directory and rerun the cget install command to pull the latest libStatGen:

rm -rf cget/
cget install -f requirements.txt
cd build && make
yutoubb commented 5 years ago

I removed everthing and tried again, but still get the same error. I don't understand why in the log 'The C/CXX compiler identification is GNU 4.4.7 but anctually I set a higher gcc in .bashrc, and if use 'gcc --version' it is 5.2.0.

$gcc --version
gcc (GCC) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Our os version is confirmed to be CentOS release 6.9 (Final).

jonathonl commented 5 years ago

I believe the binary that cmake finds first is cc and c++ (run cc --version and c++ --version), which I suspect are still the default compilers on your system.

I've tested this on CentOS 6 with default compiler and it builds fine. If it still doesn't work for you, I would suggest building libStatGen (https://github.com/statgen/libStatGen) manually and specifying it's location to Minimac4 either by cmake variables on command line or by editing the CMakelists.txt file. The following edits would work:

# Add this line after project() function to set libStatGen include path
include_directories(/path/to/libStatGen/include

# Remove this line
find_library(STATGEN_LIBRARY StatGen)

# Update this line to set absolute path to library archive
target_link_libraries(minimac4 /path/to/libStatGen/libStatGen.a ${ZLIB_LIBRARIES})
yutoubb commented 5 years ago

I changed CMakelists.txt, removed cget and build folder, and ran follow commands:

mkdir build && cd build
cmake ..
make

And I got same error. Is there still something to be changed in dep/libstatgen.cmake or some where? or should I just put libStatgen/include under src folder?

jonathonl commented 5 years ago

libstatgen.cmake is not being used with this approach. The libStatGen include and lib can be located anywhere. Did libStatGen build without error?

yutoubb commented 5 years ago

Yes, the libStatGen was built without error. Commands I used: 

git clone https://github.com/statgen/libStatGen.git 
cd libStatGen
make
jonathonl commented 5 years ago

It doesn't make sense to me that you're still getting the same error. Can you please attach the Minimac4 CMakelists.txt file that you edited?

yutoubb commented 5 years ago

The structure is:

/software/imputation/Minimac4/
├── libStatGen
│   ├── bam
│   ├── copyrights
│   ├── Doxyfile
│   ├── fastq
│   ├── general
│   ├── glf
│   ├── include
│   ├── libStatGen.a
│   ├── Makefile
│   ├── Makefiles
│   ├── README.md
│   ├── samtools
│   └── vcf
└── Minimac4
    ├── build
    ├── CMakeLists.txt
    ├── dep
    ├── install.sh
    ├── LICENSE
    ├── README.md
    ├── requirements.txt
    └── src

Content of CMakelists.txt

cmake_minimum_required(VERSION 3.2)
project(minimac4 VERSION 1.0.1)

include_directories(/software/imputation/Minimac4/libStatGen/include)

set(CMAKE_CXX_STANDARD 11)
execute_process(COMMAND date OUTPUT_VARIABLE DATE OUTPUT_STRIP_TRAILING_WHITESPACE)
execute_process(COMMAND whoami OUTPUT_VARIABLE USER OUTPUT_STRIP_TRAILING_WHITESPACE)

add_definitions(-DVERSION="${PROJECT_VERSION}" -DUSER="${USER}" -DDATE="${DATE}")

find_package(OpenMP)
if (OPENMP_FOUND)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
endif()

find_package(ZLIB REQUIRED)
add_executable(minimac4
        src/Analysis.cpp
        src/Analysis.h
        src/AnalysisChunks.cpp
        src/DosageData.cpp
        src/DosageData.h
        src/HaplotypeSet.cpp
        src/HaplotypeSet.h
        src/Imputation.cpp
        src/Imputation.h
        src/ImputationStatistics.cpp
        src/ImputationStatistics.h
        src/Main.cpp
        src/MarkovModel.cpp
        src/MarkovModel.h
        src/MarkovParameters.cpp
        src/MarkovParameters.h
        src/MyVariables.h
        src/Unique.cpp
        src/Unique.h
        src/Estimation.cpp
        src/Estimation.h)

target_link_libraries(minimac4 /software/imputation/Minimac4/libStatGen/libStatGen.a ${ZLIB_LIBRARIES})

install(TARGETS minimac4 RUNTIME DESTINATION bin)
jonathonl commented 5 years ago

I have one last theory. I bet libStatGen is being build with g++ (gcc v5.2.0) and Minimac4 is being built with c++ (gcc v4.4.7). Edit Minimac4/build/CMakeCache.txt and set CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to point to the 5.2 versions of gcc and g++ respectively. Then rebuild Minimac4. There's an ABI breakage in std::string between versions 4 and 5 of g++ that would cause such errors when linking.