search

vgteam / vg

tools for working with genome variation graphs
https://biostars.org/tag/vg/
Other
1.1k stars 194 forks source link

Cannot install on OSX #3014

Closed Ellmen closed 3 years ago

Ellmen commented 4 years ago

I am trying to install on OSX Catalina 10.15.7 using Homebrew for dependencies. Without installing GNU gcc I got stuck on an issue with building rocksdb. I tried installing GNU gcc which made it further in the install but now it can't seem to build libvgio.

End of stacktrace for . ./source_me.sh && make:

[ 76%] Building CXX object CMakeFiles/vgio.dir/src/vpkg.cpp.o
...
make[3]: *** No rule to make target `handlegraph-prefix/lib/libhandlegraph.dylib', needed by `libvgio.dylib'.  Stop.
make[2]: *** [CMakeFiles/vgio.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [lib/libvgio.a] Error 2
adamnovak commented 4 years ago

Generally we recommend using Clang on Mac; we haven't done the work necessary to convince GNU GCC to use the same standard library as the dependencies we use from Homebrew, so we can link against them.

What error did you encounter building RocksDB?

We haven't actually used RocksDB for anything that isn't more or less deprecated for a while. Maybe it's time for us to take it out.

Ellmen commented 4 years ago

For the command: . ./source_me.sh && cd deps/rocksdb && PORTABLE=1 DISABLE_JEMALLOC=1... it has a couple warnings about some directories not existing which I think are fine and then:

cache/clock_cache.d:18: *** missing separator.  Stop.
make: *** [lib/librocksdb.a] Error 2
adamnovak commented 4 years ago

Hm. Maybe delete cache/clock_cache.d and try again? The .d files are records of the dependencies that went into building something, and aren't needed except to speed up the second build by not rebuilding what shouldn't need to change.

Do you have a weird version of Clang, or a weird/old version of Make that might not be able to understand new .d files somehow? What are the contents of that file? It's supposed to end up holding some Make rules that define dependencies.

OliverPStuart commented 3 years ago

I'm not building rocksdb but I've gotten exactly the same error message as in the OP while building vg on Mojave 10.14.6 using Homebrew for dependencies and clang. Building libvgio fails and the compilation aborts.

Clang and make versions are:

$ clang --version
clang version 10.0.1 
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin

$ make --version
GNU Make 3.81
...
This program built for i386-apple-darwin11.3.0
TypAnna commented 3 years ago

I'm having the same issue as described above, using Catalina 10.15.6. I'm using clang version 8.0. and make version 3.81. Also tried clang version 12.0.0. Tried deleting cache/clock_cache.d, but did unfortunately not work. This is my error stack trace;

Makefile:155: [...]vg/include/dynamic dont exist
cache/lru_cache.d:16: *** missing separator.  Stop.
make: *** [lib/librocksdb.a] Error 2
adamnovak commented 3 years ago

Looks like it's time for one of the dev team to actually update to Catalina and its new Clang and sort this out.

adamnovak commented 3 years ago

OK, I upgraded to Catalina today, and after a bunch of fiddling to get the Xcode Command Line Tools installed properly again, I can now reproduce this issue.

adamnovak commented 3 years ago

Looks like the problem is that some of our flags meant only for our compiler invocations are wandering into RocksDB's build, and on Catalina when you combine those flags with the flags RocksDB uses, the compiler decides to write preprocessor output when it's supposed to be writing Makefile rules. Maybe it's a bug in Clang? Or a bugfix in Clang?

Either way, we don't need to be passing those flags into the RocksDB build, and I have a PR that stops it. I'm building now to see if there are any other lurking Catalina bugs.

adamnovak commented 3 years ago

OK, that PR seems to be working on Catalina for me. This should be fixed (at least with Clang) soon. We don't currently support GNU GCC on Mac, so if you want to build with that you'll need to add support to the build system yourself.

TypAnna commented 3 years ago

Hi @adamnovak ! Thank you for investigating the issue :). I have pulled the latest changes and removed the links to gcc6 (gcc and g++) in order to revert the linking I had previously done, as stated under the section (Optional) Install GNU GCC, as GNU GCC is not currently supported. However - I'm still unable to install. I get the same error as OP gets, i.e:

make[3]: *** No rule to make target `handlegraph-prefix/lib/libhandlegraph.dylib', needed by `libvgio.dylib'.  Stop.
make[2]: *** [CMakeFiles/vgio.dir/all] Error 2
make[1]: *** [all] Error 2

I'm uncertain as to why this error arises, do you happen do have any idea? Is it because GNU GCC is still used somehow, or something else that I can fix on my end?

Greatly appreciate the time you're taking!

Ellmen commented 3 years ago

I had the same issue. I also had a problem with the missing return statement in vcflib which is weird because it's addressed here https://github.com/vcflib/vcflib/commit/20ca0b81d663713f20a11b62366d7987f16b1c81 so for some reason I have an old version of vcflib (older than the commit hash in deps). I tried manually adding the return statement and then bumped into @TypAnna's problem. Thanks for the help!

adamnovak commented 3 years ago

When changing compilers, I would recommend completely deleting the vg directory and re-cloning. You could instead try make clean, which doesn't always get everything, or a more selective deletion of include, bin, lib, and deps followed by git submodule update --init --recursive to re-clone the dependencies, which should be sufficient.

The No rule to make target problem looks like a CMake problem. Someone in our lab just recently had a similar issue on Linux, and was getting:

No rule to make target 'handlegraph-prefix/lib/libhandlegraph.so', needed by 'libvgio.so'.  Stop.

This was with CMake 3.16.3, and the issue was fixed by upgrading to CMake 3.18.4. I also used CMake 3.18.4 from Macports in my Catalina Mac build testing, and that worked fine. So maybe make sure you are using that version?

@Ellmen As for making sure you have the right versions of all the submodules, you can do a git submodule update --init --recursive to make sure that the submodule versions we call for are actually checked out on disk. You might need to undo any manual edits first.

Ellmen commented 3 years ago

Just tried a fresh clone of vg with CMake 3.18.4 and got the same error

TypAnna commented 3 years ago

Unfortunately the issue (No rule to make target 'handlegraph-prefix/lib/libhandlegraph.dylib', needed by 'libvgio.dylib'. Stop.) remains after deleting the repo and then re-cloning and re-installing everything. Checked that I have CMake 3.18.4.

Ellmen commented 3 years ago

Figured it out! https://github.com/vgteam/libvgio/pull/35

adamnovak commented 3 years ago

We now need a PR against vg that pulls in the libvgio with vgteam/libvgio#35 merged to close this.

Ellmen commented 3 years ago

@adamnovak I can make a PR. The current libvgio dependency points to this https://github.com/vgteam/libvgio/pull/33 - should that get merged first?

adamnovak commented 3 years ago

Yes, it should. I just went and merged it.

On 12/9/20, Isaac Ellmen notifications@github.com wrote:

@adamnovak I can make a PR. The current libvgio dependency points to this https://github.com/vgteam/libvgio/pull/33 - should that get merged first?

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/vgteam/vg/issues/3014#issuecomment-742110460

-- Adam Novak (He/Him) Senior Software Engineer Computational Genomics Lab UC Santa Cruz Genomics Institute "Revealing life’s code."

Personal Feedback: https://forms.gle/UXZhZc123knF65Dw5