Does not build on FreeBSD

[esiefker]

I'm trying to build salmon on FreeBSD.

First issue, fetchRapMap.sh fails because it calls /bin/bash. Bash on FreeBSD is in /usr/local/bin/bash. This is easily worked around with a symlink.

Second issue: libbwa fails to build.

[ 34%] Performing build step for 'libbwa'
make[3]: "/usr/home/esiefker/salmon/salmon/external/bwa-master/Makefile" line 17: Missing dependency operator
make[3]: "/usr/home/esiefker/salmon/salmon/external/bwa-master/Makefile" line 19: Need an operator
make[3]: Fatal errors encountered -- cannot continue
make[3]: stopped in /usr/home/esiefker/salmon/salmon/external/bwa-master
*** Error code 1

In addition to failing to build bwa, bwa is present in ports and already installed on this machine. Cmake should check for this.

[esiefker]

I'm not sure whether I should create a new issue for each problem. I'll just put them all here.

BWA continued to make, when I changed the BUILD_COMMAND to use gmake. I also ran 'export MAKE=gmake', and used gmake to run the output of cmake.

BUILD_COMMAND sh -c "gmake ${QUIET_MAKE} CC=${CMAKE_C_COMPILER} CXX=${CMAKE_CXX_COMPILER}"

Unfortunately, the BWA included with Salmon is broken on FreeBSD.

kthread.c:70:2: error: unknown type name 'int64_t'

The one on Git (https://github.com/lh3/bwa) works though. So I modified DOWNLOAD_COMMAND to grab the one from Git.

DOWNLOAD_COMMAND git clone https://github.com/lh3/bwa bwa-master

Next problem, it tries to build TBB and fails. This is two problems, because I have TBB, I passed it the path, and it fails to use it. The output of cmake says:

-- Could NOT find TBB (missing: tbbmalloc_proxy) (found suitable version "2018.0", minimum required is "4.4")

tbbmalloc_proxy.h is present in /usr/local/include/tbb.

Moving on, TBB fails to build because it's hard coded to use make. I fixed this and it built.

BUILD_COMMAND gmake ${QUIET_MAKE} CXXFLAGS=${TBB_CXXFLAGS} lambdas=1 compiler=${TBB_COMPILER} cfg=release tbb_build_prefix=LIBS

Staden-io_lib fails to build with: /usr/bin/ld: undefined reference to symbol `pthread_create@@FBSD_1.0' (try adding -lthr) //lib/libthr.so.3: could not read symbols: Bad value

As advertised, -lthr fixed this issue. CONFIGURE_COMMAND ./configure --enable-shared=no --without-libcurl --prefix= LDFLAGS=-lthr CFLAGS=${LIBSTADEN_CFLAGS} CC=${CMAKE_C_COMPILER} CXX=${CMAKE_CXX_COMPILER}

This leads me to an error I can't fix on my own.

/home/esiefker/salmon/fresh-salmon/include/BWAMemStaticFuncs.hpp:36:36: error: use of undeclared identifier 'MEM_F_SELF_OVLP' int start_width = (opt->flag & MEM_F_SELF_OVLP)? 2 : 1;

There is a similar Debian bug which is fixed: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818397

I could not find the fix mentioned.

https://github.com/COMBINE-lab/salmon/commits/master?author=satta

I have done my best, but I am stuck. I hope this is helpful for those more skilled than I.

[rob-p]

Hi,

I think keeping this all in one issue is best. First, i applaud your Herculean effort! I don't have access to a FreeBSD or OpenBSD box, which is partly why we hadn't seen these issues. I can look into this, but testing is hard b/c i don't have the target environment. In the meantime, if you are just trying to use the software, perhaps give the Docker image a try (as i hear Docket can be cajoled to work on FreeBSD).

[esiefker]

Thanks. I noticed that you forked BWA. I'm guessing my substitution of mainline BWA for your forked version is behind the last error. If we get int64_t defined, that might resolve it. I'd be happy to test, and can submit some patches for my other edits to CMakefiles.txt

I will try the Docker image. I was hesitant to use it because it relies on ZFS, and I'm not sure how ZFS will interact with my jail. Probably the easier path right now though.

[apeltzer]

Same issue on Alpine Linux (latest, 3.6).

kthread.c:70:2: error: unknown type name 'int64_t'
int64_t index;
^~~~~~~
kthread.c:78:2: error: unknown type name 'int64_t'
 int64_t index;
 ^~~~~~~ 
make[3]: *** [Makefile:24: kthread.o] Error 1

[rob-p]

Hi @apeltzer,

I think this is a result of the compilers becoming stricter by default in what they accept. Code that previously compiled now wont. Here, specifically, the header inttypes.h needs to be added to that file. Could you tell me what c++ compiler you are using? This can be obtained with the output of c++ -v.

--Rob

[apeltzer]

Hi rob-p, gcc -version tells me its version 6.3.0 of the GCC compiler.

c+\+ -v
Using built-in specs.
COLLECT_GCC=c++
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-alpine-linux-musl/6.3.0/lto-wrapper
Target: x86_64-alpine-linux-musl
Configured with: /home/buildozer/aports/main/gcc/src/gcc-6.3.0/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=x86_64-alpine-linux-musl --host=x86_64-alpine-linux-musl --target=x86_64-alpine-linux-musl --with-pkgversion='Alpine 6.3.0' --enable-checking=release --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers --enable-__cxa_atexit --enable-default-pie --enable-cloog-backend --enable-languages=c,c++,objc,java,fortran,ada --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --enable-shared --enable-threads --enable-tls --with-system-zlib --with-linker-hash-style=gnu
Thread model: posix
gcc version 6.3.0 (Alpine 6.3.0)

[rob-p]

Hi @apeltzer ,

Thanks for this info. I've fixed the issues related to the bwa files in the develop branch. I tested things out using an Alpine Linux docker image to re-create your environment. Unfortunately, there seems to be a bigger issue for building from source on Alpine Linux that is actually far beyond the scope of Salmon itself. Apparently, Alpine linux uses [musl])(https://www.musl-libc.org/), a non-standard libc replacement that is missing some of the calls used by Intel's TBB. I imagine this would affect quite a bit of scientific software on this distribution. I currently haven't found a work-around, but let me know if you're aware of anything. In the mean time, does the pre-compiled binary (or the docker image) work for you?

[rob-p]

@apeltzer ,

I tried to use the pre-compiled binary myself under alpine within Docker. Initially, it suffers from the same issue (lack of a compliant libc). However, by installing the glibc compatibility layer (which was simply a few commands as laid out here), I was able to get the pre-compiled binary to work without issue. Actually, this layer may also fix building from source, but I have not checked yet.

[apeltzer]

Hi @rob-p,

I fixed it that way, adding libtbb together with your suggested libc requirement:

https://github.com/qbicsoftware/qbic-singularity-salmon/blob/5f1cc6a462d581c8605b81223fb016f47c42d110/build.sh#L8 (and then using the precompiled binaries, which is fine for us here, too)

[rob-p]

Great, @apeltzer. Thanks for letting me know!

[mmokrejs]

I just wanted to point you that Gentoo Linux also has a FreeBSD port, so the packages in Gentoo should compile on Gentoo FreeBSD too. But hey, it is just discontinued: https://wiki.gentoo.org/wiki/Gentoo_FreeBSD

I just managed to to get a working package for salmon in Gentoo (with some patches fixing the awfull curl/cmake mixture): https://github.com/gentoo/sci/tree/master/sci-biology/salmon

[outpaddling]

FYI, I'm taking another crack at an official FreeBSD port, but still hitting some gnarly issues with 1.5.2, so it might be a while.

https://github.com/outpaddling/freebsd-ports-wip/tree/master/salmon https://github.com/outpaddling/freebsd-ports-wip/tree/master/pufferfish https://github.com/COMBINE-lab/salmon/issues/502

1. The cmake system still forces downloading pufferfish during configure, which is forbidden in the ports system (like many other package managers). All downloads must occur during fetch phase and be verified using locally stored checksums. This would be easy to work around using GH_TUPLE, which downloads additional distfiles during fetch phase, except that fetchPufferfish.sh doesn't just extract the pufferfish dist, but has a long list of "cp" commands to copy pieces of it to ${INSTALL_DIR}. That's not something I'm inclined to tamper with since it will likely change with new versions and hence be a headache to maintain over time. It would be ideal if salmon could work with a separately installed pufferfish as it does with many other dependencies. This would make the port much cleaner.
2. The code is not compatible with onetbb 2021.3, which is the current FreeBSD ports version.

[outpaddling]

FYI: https://github.com/outpaddling/freebsd-ports-wip/tree/master/salmon

It would be good to update to onetbb 2021 soon. FreeBSD ports still has a 2020.3 legacy port, but we're trying to update everything that depends on it so it can be retired.

I think maybe this issue can be closed at this point?