Compiling on linux

[Juke34]

I try to make a conda recipe but for the moment I don't succeed to compile it. It compile form source, which is downloaded here https://github.com/UCLOrengoGroup/cath-tools/archive/v0.16.2.tar.gz

I get this error:

12:31:21 BIOCONDA INFO (ERR) fatal: Not a git repository (or any of the parent directories): .git 12:31:21 BIOCONDA INFO (ERR) fatal: Not a git repository (or any of the parent directories): .git 12:31:21 BIOCONDA INFO (ERR) CMake Error at CMakeLists.txt:40 (foreach): 12:31:21 BIOCONDA INFO (ERR) foreach Invalid integer: 12:31:21 BIOCONDA INFO (ERR) '/opt/conda/conda-bld/cath-tools_1586435390081/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/libgsl.so'

Any clue?

I saw that you re-wrote the CMakeList.txt, if you do a new release maybe it would be easier to compile

[sillitoe]

I haven't tried to compile this on bioconda so I'm not familiar with error messages.

The build instructions work for me on a recent Ubuntu and MacOS:

https://cath-tools.readthedocs.io/en/latest/build/

Can you confirm whether you can build this outside of bioconda?

[sillitoe]

PS might be worth chasing up the "fatal: Not a git repository.." error before you get into the compiler stuff?

[Juke34]

Here the try on bioconda: https://github.com/bioconda/bioconda-recipes/pull/21443 I didn't succeed on my Mac. I will try again Is git repo necessary to compile? I didn't look into details the CMakeLists.txt...

[tonyelewis]

Ah - that's helpful context - yes I you're hitting issues that have been fixed since the last release so I think a new release may help you. I'll do that. In the meantime, can you test this by configuring to try building from master?

One of the warnings is that you're not specifying a directory for cmake to use. It might be good to at least just put the directory at the end of the cmake command line, or ideally use CMake's -S and -B options.

It looks like your $PREFIX is evaluating to /opt/conda/conda-bld/cath-tools_1586599343268/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl which seems wrong.

[Juke34]

I'm using the last git release (v0.16.3). I will try locally with the master branch. But for bioconda we need a tarball, so I cannot use the master branch but only the releases

[tonyelewis]

0.16.3 is the new release. Any joy with that?

The PR you linked to earlier appears to refer to 0.15.2.

[Juke34]

My bad I don't know how I ended-up with version 0.15.2, I always thought I was using the 0.16.2. Thank you for pointing it. I will let you know how it goes with 0.16.2 or 0.16.3

[tonyelewis]

NP. You should now use the new 0.16.5. (There were issues with the release processes of 0.16.3 (a deployment token had expired) and 0.16.4 (I had removed some config that wasn't quite as deprecated as I'd thought)).

[Juke34]

I tried with 0.16.5 but get this error:

09:30:15 BIOCONDA INFO (OUT) [  3%] Building CXX object source/CMakeFiles/ct_chopping.dir/chopping/chopping.cpp.o
............09:42:09 BIOCONDA INFO (ERR) make[2]: *** [source/CMakeFiles/ct_uni.dir/uni/acquirer/alignment_acquirer/cora_aln_file_alignment_acquirer.cpp.o] Broken pipe

[tonyelewis]

That text (Broken pipe, interspersed messages) makes it looks like there were multiple operations occurring at the same time in a pipe and some process in the pipe died. My guess from that output is that it wasn't the make that died.

But it's hard to tell without more context. It might be good if you copy/paste more of the log(s). But actually I think it would better if you update your PR until it has the same error so we can see the full Circle-CI logs of your attempted build.

[Juke34]

Ok I updated the PR, you will have more context

[tonyelewis]

I can't see any new commits on https://github.com/Juke34/bioconda-recipes/commits/cath-tools

Have you pushed your updates?

[Juke34]

Should be fine now

[tonyelewis]

It's now failing to find libgslcblas.a, one of the GSL libraries it needs.

[Juke34]

Yes interesting, it didn't fail the same way. Locally it failed much more earlier (3%). It sounds I will not be able to test the recipe locally. Should I specify a specific PATH/VARIABLE to cmake to find the libgslcblas.a lib?

[tonyelewis]

Should I specify a specific PATH/VARIABLE to cmake to find the libgslcblas.a lib?

I suggest breaking it down into two steps:

1. Ensure that a libgslcblas.a library is correctly installed by the time make is called and that you know where it is
2. If problems persist, figure out how you can tell cmake where it is

For step 1, though it looks like you're currently specifying gsl as a requirement, (at least for Linux) I think you'll want something like libgsl-dev as described in the build documentation. On Ubuntu 18.04, this contains libgslcblas.a.

(Beware: the cath-tools CMakeLists.txt is a bit hacky in how it tries to find the libgslcblas.a by assuming it's in the same directory as the first GSL library found by FindGSL.cmake.)

[Juke34]

I checked the conda version of GSL, in version 2.5 used by defaut in the cath-tools recipe there is only libgslcblas.dylib, no libgslcblas.a while in 2.6 it's there. So I will update the recipe to see if it compile better with version >=2.6

[Juke34]

Ok now it compiles well, I forgot to put all tools in $PREFIX/bin, so build-test was not found. I re-run it. But as all built binaries are moved in $PREFIX/bin I'm not sure the test will succeed to find the build-test-data (I guess it is needed). I can save the data (all original folder if needed) in $PREFIX/share but I don't know how to tell build-test where the cath-tools original folder is standing.

[tonyelewis]

build-test looks for the data in the build-test-data directory within the directory from which it's run. So it doesn't matter where the binary is; you just need to cd to the repo root before executing build-test.

[Juke34]

Ok it is what I have done. It works locally on my Mac (build-test, I have not tried the perl test) and on conda... except the conda package didn't succeed due to time limit (5 hours). I will see what I can do, but the recipe sounds good now.

[sillitoe]

Okay great, sounds like remaining issues are outside of cath-tools I'm closing this ticket.

@tonyelewis - thanks for your time on this.

[Juke34]

As it was full of warnings and I didn't succeed to make it pass the bioconda packaging (compilation was ok ~1h, but the next packaging step ~4 hours reach time limit), I try to optimise and fix warnings. I'm trying locally on my OSX, everything seems ok excepted with BOOST. I end up with errors:

10:31:15 BIOCONDA INFO (ERR)       std::__1::basic_ostream<wchar_t, std::__1::char_traits<wchar_t> >& boost::log::v2s_mt_posix::aux::operator<<<wchar_t, std::__1::char_traits<wchar_t> >(std::__1::basic_ostream<wchar_t, std::__1::char_traits<wchar_t> >&, boost::log::v2s_mt_posix::aux::id<boost::log::v2s_mt_posix::aux::thread> const&) in libboost_log.a(thread_id.o)
10:31:15 BIOCONDA INFO (ERR)       std::__1::basic_ostream<wchar_t, std::__1::char_traits<wchar_t> >& boost::log::v2s_mt_posix::aux::operator<<<wchar_t, std::__1::char_traits<wchar_t> >(std::__1::basic_ostream<wchar_t, std::__1::char_traits<wchar_t> >&, boost::log::v2s_mt_posix::aux::id<boost::log::v2s_mt_posix::aux::process> const&) in libboost_log.a(process_id.o)
10:31:15 BIOCONDA INFO (ERR)       boost::log::v2s_mt_posix::basic_record_ostream<char>::operator<<(wchar_t const*) in libboost_log.a(record_ostream.o)
10:31:15 BIOCONDA INFO (ERR)       boost::log::v2s_mt_posix::basic_record_ostream<wchar_t>::operator<<(wchar_t const*) in libboost_log.a(record_ostream.o)
10:31:15 BIOCONDA INFO (ERR)       std::__1::basic_istream<wchar_t, std::__1::char_traits<wchar_t> >& boost::log::v2s_mt_posix::trivial::operator>><wchar_t, std::__1::char_traits<wchar_t> >(std::__1::basic_istream<wchar_t, std::__1::char_traits<wchar_t> >&, boost::log::v2s_mt_posix::trivial::severity_level&) in libboost_log.a(trivial.o)
10:31:15 BIOCONDA INFO (ERR)   "_wmemset", referenced from:
10:31:15 BIOCONDA INFO (ERR)       std::__1::ostreambuf_iterator<wchar_t, std::__1::char_traits<wchar_t> > std::__1::__pad_and_output<wchar_t, std::__1::char_traits<wchar_t> >(std::__1::ostreambuf_iterator<wchar_t, std::__1::char_traits<wchar_t> >, wchar_t const*, wchar_t const*, wchar_t const*, std::__1::ios_base&, wchar_t) in libboost_log.a(attribute_name.o)
10:31:15 BIOCONDA INFO (ERR)       std::__1::ostreambuf_iterator<wchar_t, std::__1::char_traits<wchar_t> > std::__1::__pad_and_output<wchar_t, std::__1::char_traits<wchar_t> >(std::__1::ostreambuf_iterator<wchar_t, std::__1::char_traits<wchar_t> >, wchar_t const*, wchar_t const*, wchar_t const*, std::__1::ios_base&, wchar_t) in libboost_log.a(thread_id.o)
10:31:15 BIOCONDA INFO (ERR)       std::__1::ostreambuf_iterator<wchar_t, std::__1::char_traits<wchar_t> > std::__1::__pad_and_output<wchar_t, std::__1::char_traits<wchar_t> >(std::__1::ostreambuf_iterator<wchar_t, std::__1::char_traits<wchar_t> >, wchar_t const*, wchar_t const*, wchar_t const*, std::__1::ios_base&, wchar_t) in libboost_log.a(process_id.o)
10:31:15 BIOCONDA INFO (ERR)   "_write", referenced from:
10:31:15 BIOCONDA INFO (ERR)       boost::filesystem::detail::copy_file(boost::filesystem::path const&, boost::filesystem::path const&, boost::filesystem::detail::copy_option, boost::system::error_code*) in libboost_filesystem.a(operations.o)
10:31:15 BIOCONDA INFO (ERR)      (maybe you meant: __ZN5boost10filesystem6detail15last_write_timeERKNS0_4pathEPNS_6system10error_codeE, __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIwNSt3__111char_traitsIwEENS3_9allocatorIwEEE13aligned_writeIDiEEvPKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIwNSt3__111char_traitsIwEENS3_9allocatorIwEEE13aligned_writeIDsEEvPKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIwNSt3__111char_traitsIwEENS3_9allocatorIwEEE13aligned_writeIcEEvPKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIcNSt3__111char_traitsIcEENS3_9allocatorIcEEE13aligned_writeIDiEEvPKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIwNSt3__111char_traitsIwEENS3_9allocatorIwEEE15formatted_writeIDiEERS8_PKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIwNSt3__111char_traitsIwEENS3_9allocatorIwEEE15formatted_writeEPKwl , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIwNSt3__111char_traitsIwEENS3_9allocatorIwEEE15formatted_writeIcEERS8_PKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIcNSt3__111char_traitsIcEENS3_9allocatorIcEEE15formatted_writeIDsEERS8_PKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIcNSt3__111char_traitsIcEENS3_9allocatorIcEEE13aligned_writeIDsEEvPKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIcNSt3__111char_traitsIcEENS3_9allocatorIcEEE15formatted_writeIDiEERS8_PKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIcNSt3__111char_traitsIcEENS3_9allocatorIcEEE13aligned_writeEPKcl , __ZN5boost10filesystem6detail15last_write_timeERKNS0_4pathElPNS_6system10error_codeE , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIcNSt3__111char_traitsIcEENS3_9allocatorIcEEE13aligned_writeIwEEvPKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIcNSt3__111char_traitsIcEENS3_9allocatorIcEEE15formatted_writeIwEERS8_PKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIwNSt3__111char_traitsIwEENS3_9allocatorIwEEE13aligned_writeEPKwl , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIwNSt3__111char_traitsIwEENS3_9allocatorIwEEE15formatted_writeIDsEERS8_PKT_l , __ZN5boost3log12v2s_mt_posix24basic_formatting_ostreamIcNSt3__111char_traitsIcEENS3_9allocatorIcEEE15formatted_writeEPKcl )
10:31:15 BIOCONDA INFO (ERR) ld: symbol(s) not found for architecture x86_64
10:31:15 BIOCONDA INFO (ERR) clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)
10:31:15 BIOCONDA INFO (ERR) make[2]: *** [cath-cluster] Error 1
10:31:15 BIOCONDA INFO (ERR) make[1]: *** [source/CMakeFiles/cath-cluster.dir/all] Error 2

Full log: log1.txt

Do you have any idea how to solve that?

[sillitoe]

For info, the start of that error looks like:

10:31:13 BIOCONDA INFO (ERR) ld: warning: ignoring file /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libpthread.tbd, file was built for unsupported file format ( 0x2D 0x2D 0x2D 0x20 0x21 0x74 0x61 0x70 0x69 0x2D 0x74 0x62 0x64 0x2D 0x76 0x33 ) which is not the architecture being linked (x86_64): /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libpthread.tbd
10:31:13 BIOCONDA INFO (ERR) ld: warning: ignoring file /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd, file was built for unsupported file format ( 0x2D 0x2D 0x2D 0x20 0x21 0x74 0x61 0x70 0x69 0x2D 0x74 0x62 0x64 0x2D 0x76 0x33 ) which is not the architecture being linked (x86_64): /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd
10:31:13 BIOCONDA INFO (ERR) Undefined symbols for architecture x86_64:
10:31:13 BIOCONDA INFO (ERR)   "__DefaultRuneLocale", referenced from:
10:31:13 BIOCONDA INFO (ERR)       bool boost::char_separator<char, std::__1::char_traits<char> >::operator()<std::__1::__wrap_iter<char const*>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(std::__1::__wrap_iter<char const*>&, std::__1::__wrap_iter<char const*>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) in libboost_program_options.a(options_description.o)
10:31:13 BIOCONDA INFO (ERR)       boost::optional<long> cath::common::largest_number_if_names_all_numeric_integers<cath::common::id_of_str_bidirnl>(cath::common::id_of_str_bidirnl const&)::'lambda'(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const in libct_cluster.a(old_cluster_data.cpp.o)
10:31:13 BIOCONDA INFO (ERR)   "__NSGetEnviron", referenced from:
10:31:13 BIOCONDA INFO (ERR)       boost::program_options::parse_environment(boost::program_options::options_description const&, boost::function1<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > const&) in libboost_program_options.a(parsers.o)
10:31:13 BIOCONDA INFO (ERR)   "__Unwind_Resume", referenced from:
10:31:13 BIOCONDA INFO (ERR)       _main in cath_map_clusters.cpp.o
10:31:13 BIOCONDA INFO (ERR)       cath::cath_map_clusters_program_exception_wrapper::do_run_program(int, char**) in cath_map_clusters.cpp.o
10:31:13 BIOCONDA INFO (ERR)       cath::opts::executable_options::CATH_TOOLS_CONF_FILE_SEARCH_PATH() in libct_options.a(executable_options.cpp.o)
10:31:13 BIOCONDA INFO (ERR)       cath::opts::executable_options::get_standard_usage_error_string() const in libct_options.a(executable_options.cpp.o)
10:31:13 BIOCONDA INFO (ERR)       cath::opts::executable_options::get_help_suffix_string() const in libct_options.a(executable_options.cpp.o)
10:31:13 BIOCONDA INFO (ERR)       cath::opts::executable_options::get_variables_map() const in libct_options.a(executable_options.cpp.o)
10:31:13 BIOCONDA INFO (ERR)       void boost::exception_detail::throw_exception_<cath::common::invalid_argument_exception>(cath::common::invalid_argument_exception const&, char const*, char const*, int) in libct_options.a(executable_options.cpp.o)
10:31:13 BIOCONDA INFO (ERR)       ...

[tonyelewis]

@Juke34 congratulations on getting bioconda/bioconda-recipes#21640 merged. Good work!

Are you still aiming to build this locally on your Mac? It looks like there's something substantially wrong with your building environment. I'm not very familiar with Macs but it looks like it's rejecting the pthread / standard library files it can see because they're the wrong architecture (and then it fails to find lots of pthread / standard library symbols that it needs).

I guess you need to change the compilation environment to suit those libraries or present other, compatible libraries to your compilation environment.

I'm closing this issue because this looks like a problem with your Mac build environment.

[Juke34]

When I said locally I was actually using the bioconda mulled-build on my OSX. I had to play with some flags but it works now. The recipe compile on OSX and Linux. I don't plan to install it locally, but on a cluster later with interproscan. Indeed, my plan is to make a recipe for Interproscan, and it needs cath-resolve-hits.

Thanks for your help.

Best regards

[tonyelewis]

:+1:

[tonyelewis]

@Juke34 Just to notify you… I've started changing the cath-tools build around a bit. This was triggered by problems with Travis-CI, which led me to switch to Github Actions. I'm now starting to get Conan handling as many of the dependencies as possible. I thought I should mention this in case it breaks your bioconda recipe.

If so, you might be able to get a sense of how to tweak things to get them up and running again from:

• https://cath-tools.readthedocs.io/en/latest/build/
• https://github.com/UCLOrengoGroup/cath-tools/blob/master/.github/workflows/build.yml
• https://github.com/UCLOrengoGroup/cath-tools/blob/master/Dockerfile

[Juke34]

Thank you for the feedback. I don't think I willt take care anymore of this conda recipe, but it might help others. As it is fairly used through conda (800 download/year) it might be of interest for you to continue to take care of it.