search

broadinstitute / gatk

Official code repository for GATK versions 4 and up
https://software.broadinstitute.org/gatk
Other
1.68k stars 588 forks source link

pysam compile failure #4742

Open mwalker174 opened 6 years ago

mwalker174 commented 6 years ago

On OS X El Capitan 10.11.6, when creating the conda environment I get the following error:

gcc -undefined dynamic_lookup -L/Users/markw/anaconda/envs/gatk/lib -L/Users/markw/anaconda/envs/gatk/lib -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/pysam/libchtslib.o build/temp.macosx-10.7-x86_64-3.6/pysam/htslib_util.o build/temp.macosx-10.7-x86_64-3.6/htslib/kfunc.o build/temp.macosx-10.7-x86_64-3.6/htslib/knetfile.o build/temp.macosx-10.7-x86_64-3.6/htslib/kstring.o build/temp.macosx-10.7-x86_64-3.6/htslib/bcf_sr_sort.o build/temp.macosx-10.7-x86_64-3.6/htslib/bgzf.o build/temp.macosx-10.7-x86_64-3.6/htslib/errmod.o build/temp.macosx-10.7-x86_64-3.6/htslib/faidx.o build/temp.macosx-10.7-x86_64-3.6/htslib/hfile.o build/temp.macosx-10.7-x86_64-3.6/htslib/hfile_net.o build/temp.macosx-10.7-x86_64-3.6/htslib/hts.o build/temp.macosx-10.7-x86_64-3.6/htslib/hts_os.o build/temp.macosx-10.7-x86_64-3.6/htslib/md5.o build/temp.macosx-10.7-x86_64-3.6/htslib/multipart.o build/temp.macosx-10.7-x86_64-3.6/htslib/probaln.o build/temp.macosx-10.7-x86_64-3.6/htslib/realn.o build/temp.macosx-10.7-x86_64-3.6/htslib/regidx.o build/temp.macosx-10.7-x86_64-3.6/htslib/sam.o build/temp.macosx-10.7-x86_64-3.6/htslib/synced_bcf_reader.o build/temp.macosx-10.7-x86_64-3.6/htslib/vcf_sweep.o build/temp.macosx-10.7-x86_64-3.6/htslib/tbx.o build/temp.macosx-10.7-x86_64-3.6/htslib/textutils.o build/temp.macosx-10.7-x86_64-3.6/htslib/thread_pool.o build/temp.macosx-10.7-x86_64-3.6/htslib/vcf.o build/temp.macosx-10.7-x86_64-3.6/htslib/vcfutils.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/cram_codecs.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/cram_decode.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/cram_encode.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/cram_external.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/cram_index.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/cram_io.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/cram_samtools.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/cram_stats.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/files.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/mFILE.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/open_trace_file.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/pooled_alloc.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/rANS_static.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/sam_header.o build/temp.macosx-10.7-x86_64-3.6/htslib/cram/string_alloc.o build/temp.macosx-10.7-x86_64-3.6/htslib/hfile_libcurl.o build/temp.macosx-10.7-x86_64-3.6/htslib/hfile_gcs.o build/temp.macosx-10.7-x86_64-3.6/htslib/hfile_s3.o -Lpysam -L. -Lbuild/lib.macosx-10.7-x86_64-3.6/pysam -Lbuild/lib.macosx-10.7-x86_64-3.6/pysam -L/Users/markw/anaconda/envs/gatk/lib -lz -llzma -lbz2 -lz -lcurl -o build/lib.macosx-10.7-x86_64-3.6/pysam/libchtslib.cpython-36m-darwin.so -dynamiclib -rpath @loader_path -Wl,-headerpad_max_install_names -Wl,-install_name,@rpath/libchtslib.cpython-36m-darwin.so -Wl,-x
    gcc: error: @loader_path: No such file or directory
    gcc: error: unrecognized command line option ‘-rpath’
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/Users/markw/anaconda/envs/gatk/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/x6/k5tc9mwd2z7dm1dthy4hv9nxmt8q9j/T/pip-build-aybz1ucp/pysam/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/x6/k5tc9mwd2z7dm1dthy4hv9nxmt8q9j/T/pip-z_8e2y_r-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/x6/k5tc9mwd2z7dm1dthy4hv9nxmt8q9j/T/pip-build-aybz1ucp/pysam/

CondaValueError: pip returned an error

This appears to have been an issue with pysam in the past: https://github.com/pysam-developers/pysam/issues/323

samuelklee commented 6 years ago

What is the timeline for removing pysam, PyVCF, etc. dependencies (#4465)? I think we agreed to abide by certain rules when we started python development, but I don't know who is responsible for enforcing them...

cmnbroad commented 6 years ago

I think removing pysam is dependent on writing Java front ends for the CNN tools that are currently just pass-through to python (https://github.com/broadinstitute/gatk/issues/4533, https://github.com/broadinstitute/gatk/issues/4534 and https://github.com/broadinstitute/gatk/issues/4535). @lucidtronix Do you have a timeframe for that work ?

lucidtronix commented 6 years ago

Sorry, just saw this now. We still don't have a simple solution for training models without pysam. We can probably do something similar to what we do with inference, but I think the current priority is to improve inference throughput so it will probably be a little while before we get to re-writing the training code. If people feel we should re-prioritize please let me know. I have installed the conda environment on the same OSX version, without seeing this issue. Which gcc version are you using @mwalker174 ?
My gcc -v output is:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
mwalker174 commented 6 years ago

@lucidtronix Probably something's up with my configuration... $ gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/Users/markw/anaconda/envs/py27/bin/../libexec/gcc/x86_64-apple-darwin11.4.2/4.8.5/lto-wrapper Target: x86_64-apple-darwin11.4.2 Configured with: ./configure --prefix=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac --with-gxx-include-dir=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/gcc/include/c++ --bindir=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/bin --datarootdir=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/share --libdir=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib --with-gmp=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac --with-mpfr=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac --with-mpc=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac --with-isl=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac --with-cloog=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac --with-boot-ldflags='-Wl,-headerpad_max_install_names -Wl,-L/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib -Wl,-L/usr/lib' --with-stage1-ldflags='-Wl,-headerpad_max_install_names -Wl,-L/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib -Wl,-L/usr/lib' --enable-checking=release --with-tune=generic --disable-multilib Thread model: posix gcc version 4.8.5 (GCC) Perhaps using the Xcode version would fix things?

lucidtronix commented 6 years ago

That would be my first guess too. Let me know if it doesn't.