Make fetching all dependencies optional like boost

[flying-sheep]

My OS has libbwa and cmph packaged, and I’d rather use those versions than having to read that

error: downloading 'http://www.cs.stonybrook.edu/~rp/files/bwa-master.tar.gz' failed

[roryk]

+1 for this, libsparsehash fails to build on yosemite which prevents the homebrew-science formula from building: https://github.com/Homebrew/homebrew-science/pull/2706

[rob-p]

Hi @roryk, I'm surprised that libsparsehash fails on yosemite, as that's the OSX version I'm working on and the one on which I create the pre-built executable. In fact, I pull from our own modified branch of sparsehash that has fixes specifically aimed at OSX. Can you tell me what errors, specifically, you're seeing on Yosemite? For reference, I'm building with Apples provided clang compiler:

Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.5.0
Thread model: posix

As a side note, this question isn't meant to suggest that I'm not in favor of making dependency fetching optional (I am), but rather I'm also interested in figuring out why you might be experiencing a build failure on a system where I've built successfully before.

[roryk]

Thanks @rob-p,

Apple LLVM version 7.0.0 (clang-700.0.72)
Target: x86_64-apple-darwin14.5.0
Thread model: posix

I wasn't having any troubles with 6.1.

Here is a gist of the log: https://gist.github.com/roryk/fda5c55fb732ea06a1b5

And related for the google-sparsehash formula: https://github.com/Homebrew/homebrew/issues/41087

[rob-p]

Ahh --- I haven't upgraded to 7.0.0 yet. I'll try and take a look at this soon; thanks for alerting me to it!

[roryk]

Upgrading Xcode always reminds me of this:

https://www.jwz.org/blog/2012/12/is-there-a-long-german-word-for/

[flying-sheep]

all of this doesn’t matter much. i want to be able to install this if all dependencies are there without it trying to download stuff.

[rob-p]

Hi @flying-sheep,

While I do think that the regression in XCode "matters", you're absolutely right that it's orthogonal the the original issue. You're also also right, of course, that libraries already present on the host OS shouldn't need to be downloaded (unless, like sparsehash, they contain special updates, modifications or fixes that are not present in an upstream version of the package). I'll try to complete this request this week.

[roryk]

Looks like google-sparsehash might have a fix for the 7.0.0 issue:

https://github.com/sparsehash/sparsehash/issues/111

[rob-p]

Fantastic! I was just getting around to working on this. It actually looks like, after the update (when I pulled out all of the libraries that are no longer used by Sailfish after the update and split with salmon --- e.g. libbwa), there are actually few dependencies whose fetching should be optional. Right now, the only things we fetch unconditionally are Jellyfish, libgff, and sparsehash. This is because:

• We need the dev files for Jellyfish (including the config.h file, since we use it as a library)
• LibGFF isn't "a thing" --- i.e. it's a small project I created specifically for the purposes of library-izing the gff parser used by Cufflinks, and thus there's no good reason to expect the user would have it pre-installed or to check for it
• We have to use the version of sparsehash with all of the relevant patches applied to avoid build failure on OSX.

For any other dependencies, we now check if they are installed (can be found via the relevant CMake findPackage) before we fetch them.

@roryk --- I've merged the relevant changes to build on Clang 7.0 upstream. Can you try a clean build and see if it works for you? You'll have to make sure you've gotten rid of all of CMake's old junk so that it doesn't attempt to use any cached files.

[rob-p]

Of course, let me un-botch the Travis-CI build first ... Sigh.

[rob-p]

Ok --- fixed (there were some small changes with the latest jellyfish that I had to adjust for).

[roryk]

Thanks @rob-p, builds fine and the test passes under 7.0.0 now. Sorry to hijack this issue.

[roryk]

Could you push a new release so I can fix the homebrew-science formula? :)

[rob-p]

Done :).

[sjackman]

:+1: I've updated the Homebrew-science formula to 0.9.2.

[mmokrejs]

I hit this issue (automated downloads are being run) as well with current git master.

build.log.txt

I have dev-cpp/sparsehash-2.0.3-r1, dev-libs/libdivsufsort-2.0.1. I have no idea where to get RapMap and libgff (just kidding, am lazy to read the code where the curl downloads are defined).

Please upgrade Readme.md with the requirements and URLs where to get them from.

[sjackman]

You could try installing Sailfish using either Linuxbrew or Bioconda.

[mmokrejs]

Well I am trying to move ahead with the package for Gentoo Linux so it does not really help. Bundled/autodownloaded approach could remain for users who could even download a bunch of binaries and libs (they do not need the sources either) but the source tarball should be just clean.