3.1.1 build failure on macOS and Linux

[mjsteinbaugh]

Hi sra-tools team,

I'm hitting a build failure for 3.1.1 on macOS:

gmake[2]: Entering directory '/private/var/folders/9b/4gh0pghx1b71jjd0wjh5mj880000gn/T/tmp.Q2H9UDvhvN/src-cmake-bd8446b0e4'
[ 96%] Building CXX object tools/external/driver-tool/CMakeFiles/sratools.dir/sratools.cpp.o
cd /private/var/folders/9b/4gh0pghx1b71jjd0wjh5mj880000gn/T/tmp.Q2H9UDvhvN/src-cmake-bd8446b0e4/tools/external/driver-tool && /Library/Developer/CommandLineTools/usr/bin/c++ -D" -DBMNEONOPT -DBSD -DBUILD_PATH=\"/private/var/folders/9b/4gh0pghx1b71jjd0wjh5mj880000gn/T/tmp.Q2H9UDvhvN/src-cmake-bd8446b0e4/bin\" -DMAC -DNDEBUG -DPREFIX_PATH=\"/opt/koopa/app/libxml2/2.12.3 -DUNIX -D_ARCH_BITS=64 -D__mod__=\"tools/driver-tool\" -Darm64 -I/opt/koopa/app/ncbi-vdb/3.1.1/include -I/opt/koopa/app/ncbi-vdb/3.1.1/include/cc/clang -I/opt/koopa/app/ncbi-vdb/3.1.1/include/cc/clang/arm64 -I/opt/koopa/app/ncbi-vdb/3.1.1/include/os/mac -I/opt/koopa/app/ncbi-vdb/3.1.1/include/os/unix -I/private/var/folders/9b/4gh0pghx1b71jjd0wjh5mj880000gn/T/tmp.Q2H9UDvhvN/src/ngs/ngs-sdk -I/private/var/folders/9b/4gh0pghx1b71jjd0wjh5mj880000gn/T/tmp.Q2H9UDvhvN/src/libs/inc -Wall   -I/opt/koopa/app/icu4c/74.2/include -I/opt/koopa/app/libxml2/2.12.3/include/libxml2 -Wno-unused-function -Wno-tautological-undefined-compare -O3 -DNDEBUG -std=c++11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -MD -MT tools/external/driver-tool/CMakeFiles/sratools.dir/sratools.cpp.o -MF CMakeFiles/sratools.dir/sratools.cpp.o.d -o CMakeFiles/sratools.dir/sratools.cpp.o -c /private/var/folders/9b/4gh0pghx1b71jjd0wjh5mj880000gn/T/tmp.Q2H9UDvhvN/src/tools/external/driver-tool/sratools.cpp
/bin/sh: -c: line 0: unexpected EOF while looking for matching `"'
/bin/sh: -c: line 1: syntax error: unexpected end of file
gmake[2]: *** [tools/external/driver-tool/CMakeFiles/sratools.dir/build.make:79: tools/external/driver-tool/CMakeFiles/sratools.dir/sratools.cpp.o] Error 2
gmake[2]: Leaving directory '/private/var/folders/9b/4gh0pghx1b71jjd0wjh5mj880000gn/T/tmp.Q2H9UDvhvN/src-cmake-bd8446b0e4'
gmake[1]: *** [CMakeFiles/Makefile2:2920: tools/external/driver-tool/CMakeFiles/sratools.dir/all] Error 2
gmake[1]: Leaving directory '/private/var/folders/9b/4gh0pghx1b71jjd0wjh5mj880000gn/T/tmp.Q2H9UDvhvN/src-cmake-bd8446b0e4'
gmake: *** [Makefile:169: all] Error 2

I can confirm that 3.1.0 builds as expected, this seems to be specific to the 3.1.1 update.

My build script is here, for reference: https://github.com/acidgenomics/koopa/tree/develop/lang/bash/include/install/common/shared/ncbi-sra-tools.sh

Best, Mike

[mjsteinbaugh]

Here's related build update for Homebrew: https://github.com/Homebrew/homebrew-core/pull/172428

[durbrow]

How are you building the toolkit? If you are using something like Homebrew, we have no means to debug it.

[mjsteinbaugh]

@durbrow Building from source without Homebrew.

Can confirm that this issue is reproducible on both Apple Silicon and Intel Macs.

I'm having trouble figuring out where this error is occuring during the build:

/bin/sh: -c: line 0: unexpected EOF while looking for matching `"'
/bin/sh: -c: line 1: syntax error: unexpected end of file

Here's a reprex using Docker, running Debian instead of macOS:

docker run -it public.ecr.aws/acidgenomics/koopa

koopa system switch-to-develop
koopa update koopa
koopa configure system base
koopa install ncbi-sra-tools

[mjsteinbaugh]

Seeing a similar issue on Debian for 3.1.1, whereas 3.1.0 builds successfully:

[ 96%] Building CXX object tools/external/driver-tool/CMakeFiles/sratools.dir/sratools.cpp.o
cd /tmp/tmp.ECxBOhs39Q/src-cmake-7feb12b20d/tools/external/driver-tool && /usr/bin/c++ -D" -DBMNEONOPT -DBUILD_PATH=\"/tmp/tmp.ECxBOhs39Q/src-cmake-7feb12b20d/bin\" -DLINUX -DNDEBUG -DPREFIX_PATH=\"/opt/koopa/app/libxml2/2.12.3 -DUNIX -D_ARCH_BITS=64 -D_GNU_SOURCE -D__float128=_Float128 -D__mod__=\"tools/driver-tool\" -Darm64 -I/opt/koopa/app/ncbi-vdb/3.1.1/include -I/opt/koopa/app/ncbi-vdb/3.1.1/include/cc/gcc -I/opt/koopa/app/ncbi-vdb/3.1.1/include/cc/gcc/arm64 -I/opt/koopa/app/ncbi-vdb/3.1.1/include/os/linux -I/opt/koopa/app/ncbi-vdb/3.1.1/include/os/unix -I/tmp/tmp.ECxBOhs39Q/src/ngs/ngs-sdk -I/tmp/tmp.ECxBOhs39Q/src/libs/inc -Wall   -I/opt/koopa/app/icu4c/74.2/include -I/opt/koopa/app/libxml2/2.12.3/include/libxml2 -Wno-unused-function  -O3 -DNDEBUG -std=c++11   -rdynamic -MD -MT tools/external/driver-tool/CMakeFiles/sratools.dir/sratools.cpp.o -MF CMakeFiles/sratools.dir/sratools.cpp.o.d -o CMakeFiles/sratools.dir/sratools.cpp.o -c /tmp/tmp.ECxBOhs39Q/src/tools/external/driver-tool/sratools.cpp
/bin/sh: 1: Syntax error: Unterminated quoted string
gmake[2]: *** [tools/external/driver-tool/CMakeFiles/sratools.dir/build.make:79: tools/external/driver-tool/CMakeFiles/sratools.dir/sratools.cpp.o] Error 2
gmake[2]: Leaving directory '/tmp/tmp.ECxBOhs39Q/src-cmake-7feb12b20d'
gmake[1]: Leaving directory '/tmp/tmp.ECxBOhs39Q/src-cmake-7feb12b20d'
gmake[1]: *** [CMakeFiles/Makefile2:2920: tools/external/driver-tool/CMakeFiles/sratools.dir/all] Error 2
gmake: *** [Makefile:169: all] Error 2

[durbrow]

What are your build commands?

In house, we use our configure scripts (which are in the code repo) and make. This works for us on Linux, FreeBSD, Windows, and on Intel and ARM Macs.

At home, I use pure cmake to configure and build on both Intel and ARM Macs and on Alpine Linux in docker.

I just now tried on an M1 Mac-mini:

> cd ${TMPDIR}
> git clone https://github.com/ncbi/ncbi-vdb.git
> git clone https://github.com/ncbi/sra-tools.git
> mkdir build
> cd build
> cmake -S "$(cd ../ncbi-vdb; pwd)" -B ncbi-vdb
> cmake --build ncbi-vdb
> cmake -D VDB_LIBDIR="${PWD}/ncbi-vdb/lib" -D CMAKE_INSTALL_PREFIX="${PWD}/sratoolkit" -S "$(cd ../sra-tools; pwd)" -B sra-tools 
> cmake --build sra-tools --target install
> ./sratoolkit/bin/prefetch --version

./sratoolkit/bin/prefetch : 3.1.1

[mjsteinbaugh]

@durbrow , thanks the CMake example above was a helpful reference. Building ncbi-vdb and sra-tools side-by-side in the same directory seems to have resolved my issue.

My current build script is here for reference: https://github.com/acidgenomics/koopa/blob/main/lang/bash/include/install/common/shared/ncbi-sra-tools.sh