Install Request: SRA Toolkit and dependencies on Myriad [IN06055391] [IN06072518]

[balston]

Home page: https://github.com/ncbi/sra-tools

Installation from source: https://github.com/ncbi/sra-tools/wiki/Building-and-Installing-from-Source

[balston]

Also need this:

https://github.com/ncbi/ncbi-vdb

and HDF5?

[balston]

I have a first attempt at a build script which I'm now running on Myriad from the ccspapp account:

module -f unload compilers mpi
./SRA-tools-3.0.6_install 2>&1 | tee ~/Software/SRA-tools/SRA-tools-3.0.6_install.log-1

[balston]

It didn't get very far. It is picking up the wrong version of cmake. It used the version in /usr/bin and not the module one?

[balston]

Fixed the CMAKE issue so its using the version from the cmake/3.21.1 module. I'm now getting the following error:

configured with: "'--prefix=/shared/ucl/apps/ncbi/ncbi-vdb/3.0.6/gnu-4.9.2' '--with-hdf5-prefix=/shared/ucl/apps/hdf/5-1.10.5/gnu-4.9.2' '--with-xml2-prefix=/shared/ucl/apps/libxml2/2.9.4/gnu-4.9.2'"
mkdir -p /home/ccspapp/ncbi-outdir/ncbi-vdb/linux/gcc/x86_64/rel/obj
cd /home/ccspapp/ncbi-outdir/ncbi-vdb/linux/gcc/x86_64/rel/obj; cmake -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=/home/ccspapp/ncbi-outdir/ncbi-vdb/linux/gcc/x86_64/rel/ilib -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/ccspapp/ncbi-outdir/ncbi-vdb/linux/gcc/x86_64/rel/lib -DCMAKE_RUNTIME_OUTPUT_DIRECTORY=/home/ccspapp/ncbi-outdir/ncbi-vdb/linux/gcc/x86_64/rel/bin -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/shared/ucl/apps/ncbi/ncbi-vdb/3.0.6/gnu-4.9.2 -DINST_BINDIR=/shared/ucl/apps/ncbi/ncbi-vdb/3.0.6/gnu-4.9.2/bin -DINST_LIBDIR=/shared/ucl/apps/ncbi/ncbi-vdb/3.0.6/gnu-4.9.2/lib64 -DINST_INCDIR=/shared/ucl/apps/ncbi/ncbi-vdb/3.0.6/gnu-4.9.2/include -DCMAKE_C_COMPILER=/bin/gcc -DCMAKE_CXX_COMPILER=/bin/g++ -DPYTHON_PATH=/shared/ucl/apps/python/bundles/python39-6.0.0/venv/bin/python -DRUN_SANITIZER_TESTS= -DRUN_SANITIZER_TESTS_OVERRIDE=  -DLIBS_ONLY=ON /dev/shm/tmp.JB6v01bUWQ/ncbi-vdb-3.0.6
CMake Error: The source "/dev/shm/tmp.JB6v01bUWQ/ncbi-vdb-3.0.6/CMakeLists.txt" does not match the source "/dev/shm/tmp.WsX6wcJ0tP/ncbi-vdb-3.0.6/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.
make: *** [cmake_libs] Error 1

Has it picked up something from the first failed attempt?

[balston]

Yes reading the output log a bit more carefully the build process is writing files to here:

/home/ccspapp/ncbi-outdir/ncbi-vdb/linux/gcc/x86_64/rel/obj

and there is a CMakeCache.txt cache file here. This needs to be deleted at that start of every build. Updating the build script accordingly.

[balston]

I managed to resolve the CMAKE problems now.

[balston]

My latest attempt to build failed with compilation errors but looking at the logs:

checking for gcc... gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)

it's picking up the wrong version of gcc ignoring the loaded module.

[balston]

I've sorted out it picking up the wrong compilers by editing after the two config steps the resulting Makefile.env files in the two build directories. Setting both -DCMAKE_C_COMPILER and -DCMAKE_CXX_COMPILER to the correct locations.

The first half now builds OK but the second doesn't. C++ compilation errors.

I'm switch to the GNU 10.2.0 compilers to see if that solves the issue.

[balston]

Switching to GNU 10.2.0 compilers seems to have resolved the issue.

I've been building in my Scratch so will now have to update the real build script and run from ccspapp to install in /shared/ucl/apps.

[balston]

build script updated and ready to test from ccspapp.

[balston]

Trying my updated build script from ccspapp:

module -f unload compilers mpi gcc-libs
module load beta-modules
 ./SRA-tools-3.0.6_install 2>&1 | tee ~/Software/SRA-tools/SRA-tools-3.0.6_install.log-1

[balston]

almost working ... I made a mistake in the HDF5 setup. Updating and running again ...

[balston]

ok I think the build script has finally worked.

Next step produce a suitable module file.

[balston]

module file uploaded onto Myriad. To access SRA Tools use the following module commands:

module -f unload compilers mpi gcc-libs
module load beta-modules
module load gcc-libs/10.2.0
module load compilers/gnu/10.2.0
module load java/openjdk-11/11.0.1
module load hdf/5-1.10.6/gnu-10.2.0
module load libxml2/2.9.4/gnu-4.9.2
module load python3/recommended
module load sra-tools/3.0.6/gnu-10.2.0

[balston]

users have been informed.