search

gkno / gkno_launcher

The gkno launcher for executing tools or pipelines
MIT License
32 stars 7 forks source link

OS X build fails #67

Open bede opened 8 years ago

bede commented 8 years ago

GKNO fails to build for me on Mac OS 10.10.5 due to Jellyfish it seems.

Any suggestions?

Thanks, Bede

l-umobcdja2:gkno_launcher Bede$ ./gkno build

========================================================
  University of Utah gkno package

  version:    2.50.0
  date:       January 2016
  git commit: 71c5dfa0808db6e56cc9ac08357be060707bfb2f
========================================================

Checking dependencies...done.
Initializing component data...done.
Building tools: 
  bamaddrg...done.
  bamtools...done.
  bedtools...done.
  blast...done.
  bwa...done.
  freebayes...ERROR: See logs/build_freebayes.* files for more details.
  htslib...done.
  samtools...done.
  pindel...ERROR: See logs/build_pindel.* files for more details.
  jellyfish...Traceback (most recent call last):
  File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno.py", line 430, in <module>
    main()
  File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno.py", line 102, in main
    if admin.run(sys.argv):
  File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 86, in run
    if   self.mode == "build"       : success = self.build()
  File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 155, in build
    if self.buildTool(tool):
  File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 560, in buildTool
    success = tool.build(out, err)
  File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 73, in build
    return self.doBuild()
  File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 361, in doBuild
    if not self.runCommand("autoreconf -i") : return False
  File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 90, in runCommand
    p = subprocess.Popen(command.split(), env=self.environ, stdout=self.out, stderr=self.err)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
AlistairNWard commented 8 years ago

Hey Bede,

Have you tried just running gkno build again. Sometimes it squeaks through the second time. I'm at a conference this week, so have limited time, but I think I'm going to just remove Jellyfish completely from gkno for now. It has been a constant thorn in my side and isn't used in any of the major pipelines at the moment anyway. I'll let you know when I've made the update. There is a thread on the Freebayes github page to discuss compilation on a Mac. Apparently there have been some recent changes to Mac OS that force clang and I haven't looked into putting instructions for Mac compilation there yet. You might want to take a look here:

https://groups.google.com/forum/#!topic/freebayes/wWa5zUK0zLc

Please let me know of other problems you encounter or if you want any tips on running gkno or have suggestions. They're always welcome.

Alistair Ward, PhD Director, Research and Science Department of Human Genetics USTAR Center for Genetic Discovery Eccles Institute of Human Genetics University of Utah School of Medicine​ 15 North 2030 East, Room 7140 Salt Lake City, UT 84112-5330

Co-founder, Frameshift labs, LLC

On Thu, Mar 10, 2016 at 3:28 AM, Bede Constantinides < notifications@github.com> wrote:

GKNO fails to build for me on Mac OS 10.10.5 due to Jellyfish it seems.

Any suggestions?

Thanks, Bede

l-umobcdja2:gkno_launcher Bede$ ./gkno build

University of Utah gkno package

version: 2.50.0 date: January 2016

git commit: 71c5dfa0808db6e56cc9ac08357be060707bfb2f

Checking dependencies...done. Initializing component data...done. Building tools: bamaddrg...done. bamtools...done. bedtools...done. blast...done. bwa...done. freebayes...ERROR: See logs/build_freebayes.* files for more details. htslib...done. samtools...done. pindel...ERROR: See logs/build_pindel.* files for more details. jellyfish...Traceback (most recent call last): File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno.py", line 430, in main() File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno.py", line 102, in main if admin.run(sys.argv): File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 86, in run if self.mode == "build" : success = self.build() File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 155, in build if self.buildTool(tool): File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 560, in buildTool success = tool.build(out, err) File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 73, in build return self.doBuild() File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 361, in doBuild if not self.runCommand("autoreconf -i") : return False File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 90, in runCommand p = subprocess.Popen(command.split(), env=self.environ, stdout=self.out, stderr=self.err) File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in init errread, errwrite) File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

— Reply to this email directly or view it on GitHub https://github.com/gkno/gkno_launcher/issues/67.

AlistairNWard commented 8 years ago

Hey Bede, I updated gkno to version 2.52.0 with Jellyfish removed, so you can reclone and try again. Please let me know if you have any other problems.

By the way, there are a lot of options in gkno that make it easy to run lots of things that I haven't had a chance to get up on the website, so feel free to ask questions.

Alistair Ward, PhD Director, Research and Science Department of Human Genetics USTAR Center for Genetic Discovery Eccles Institute of Human Genetics University of Utah School of Medicine​ 15 North 2030 East, Room 7140 Salt Lake City, UT 84112-5330

Co-founder, Frameshift labs, LLC

On Thu, Mar 10, 2016 at 2:42 PM, Alistair Ward alistairnward@gmail.com wrote:

Hey Bede,

Have you tried just running gkno build again. Sometimes it squeaks through the second time. I'm at a conference this week, so have limited time, but I think I'm going to just remove Jellyfish completely from gkno for now. It has been a constant thorn in my side and isn't used in any of the major pipelines at the moment anyway. I'll let you know when I've made the update. There is a thread on the Freebayes github page to discuss compilation on a Mac. Apparently there have been some recent changes to Mac OS that force clang and I haven't looked into putting instructions for Mac compilation there yet. You might want to take a look here:

https://groups.google.com/forum/#!topic/freebayes/wWa5zUK0zLc

Please let me know of other problems you encounter or if you want any tips on running gkno or have suggestions. They're always welcome.

Alistair Ward, PhD Director, Research and Science Department of Human Genetics USTAR Center for Genetic Discovery Eccles Institute of Human Genetics University of Utah School of Medicine​ 15 North 2030 East, Room 7140 Salt Lake City, UT 84112-5330

Co-founder, Frameshift labs, LLC

On Thu, Mar 10, 2016 at 3:28 AM, Bede Constantinides < notifications@github.com> wrote:

GKNO fails to build for me on Mac OS 10.10.5 due to Jellyfish it seems.

Any suggestions?

Thanks, Bede

l-umobcdja2:gkno_launcher Bede$ ./gkno build

University of Utah gkno package

version: 2.50.0 date: January 2016

git commit: 71c5dfa0808db6e56cc9ac08357be060707bfb2f

Checking dependencies...done. Initializing component data...done. Building tools: bamaddrg...done. bamtools...done. bedtools...done. blast...done. bwa...done. freebayes...ERROR: See logs/build_freebayes.* files for more details. htslib...done. samtools...done. pindel...ERROR: See logs/build_pindel.* files for more details. jellyfish...Traceback (most recent call last): File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno.py", line 430, in main() File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno.py", line 102, in main if admin.run(sys.argv): File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 86, in run if self.mode == "build" : success = self.build() File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 155, in build if self.buildTool(tool): File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/adminUtils.py", line 560, in buildTool success = tool.build(out, err) File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 73, in build return self.doBuild() File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 361, in doBuild if not self.runCommand("autoreconf -i") : return False File "/Users/Bede/Desktop/uttl/gkno_launcher/src/gkno/conf/gknoTools.py", line 90, in runCommand p = subprocess.Popen(command.split(), env=self.environ, stdout=self.out, stderr=self.err) File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in init errread, errwrite) File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

— Reply to this email directly or view it on GitHub https://github.com/gkno/gkno_launcher/issues/67.

bede commented 8 years ago

Hi @AlistairNWard , thanks for the reply. This is how things are looking for me as of today. I've not had a chance to run this on another Mac, but consider this one in pretty good shape… It's running a fairly standard setup with default compiler config etc. and additonal compilers installed unintrusively via brew

FreeBayes build output: https://gist.github.com/bede/fb8683029c6c10d4ad99a015c9fc8b7b

$ git clone https://github.com/gkno/gkno_launcher.git
Cloning into 'gkno_launcher'...
remote: Counting objects: 9618, done.
remote: Compressing objects: 100% (31/31), done.
remote: Total 9618 (delta 13), reused 0 (delta 0), pack-reused 9587
Receiving objects: 100% (9618/9618), 113.08 MiB | 4.01 MiB/s, done.
Resolving deltas: 100% (6853/6853), done.
Checking connectivity... done.
$ cd gkno_launcher
$ ./gkno build

========================================================
  University of Utah gkno package

  version:    2.53.5
  date:       May 2016
  git commit: 394ac2de05295c659adfa19d5db4d15a2a6d43c0
========================================================

Checking dependencies...done.
Initializing component data...done.
Building tools: 
  bamaddrg...done.
  bamtools...done.
  bedtools...done.
  blast...done.
  bwa...done.
  freebayes...ERROR: See logs/build_freebayes.* files for more details.
  htslib...done.
  samtools...done.
  pindel...ERROR: See logs/build_pindel.* files for more details.
  libStatGen...done.
  bamUtil...done.
  fastQValidator...done.
  qplot...done.
  verifyBamID...done.
  mosaik...ERROR: See logs/build_mosaik.* files for more details.
  mutatrix...ERROR: See logs/build_mutatrix.* files for more details.
  picard...done.
  samblaster...done.
  seqan...
  snpEff...done.
  tabix...done.
  tangram...done.
  vcflib...ERROR: See logs/build_vcflib.* files for more details.
  vt...done.
  wham...ERROR: See logs/build_wham.* files for more details.
Fetching default resources:
  tutorial:
    Downloading files... 100%
    Unpacking files...done.

WARNING: Not all tools were built successfully.

DETAILS: While building gkno, some of the tools failed to compile successfully. Please check the
         log files to determine the cause of the failures and rebuild. In the meantime, all
         other tools and pipelines not containing the failed tools can be used. The tools that
         failed to build are:

          pindel
          vcflib
          freebayes
          wham
          mosaik
          mutatrix
$
AlistairNWard commented 8 years ago

Hey Bede,

Thanks for continuing to keep me posted. Unfortunately, I don't seem to be able to get Freebayes to compile on my Mac either. I don't think the other software that fails to compile is such a big issue, since they are more niche applications. Until we can get a general compilation solution for Freebayes (vcflib) on the Mac, gkno is pretty much a non-Mac Linux application for now. I'll keep looking into this though and I'll let you know if I can get Freebayes to compile consistently on the Mac and then provide instructions for gkno as well.

Al

Alistair Ward, PhD Director, Research and Science Department of Human Genetics USTAR Center for Genetic Discovery Eccles Institute of Human Genetics University of Utah School of Medicine​ 15 North 2030 East, Room 7140 Salt Lake City, UT 84112-5330

Co-founder, Frameshift labs, LLC

On Fri, May 20, 2016 at 7:11 AM, Bede Constantinides < notifications@github.com> wrote:

Hi @AlistairNWard https://github.com/AlistairNWard , thanks for the reply. This is how things are looking for me as of today. I've npt had a chance to run this on another Mac. I consider this Mac in pretty good shape... It's running a pretty standard setup with default compiler config etc.

FreeBayes build output: https://gist.github.com/bede/fb8683029c6c10d4ad99a015c9fc8b7b

$ git clone https://github.com/gkno/gkno_launcher.git Cloning into 'gkno_launcher'... remote: Counting objects: 9618, done. remote: Compressing objects: 100% (31/31), done. remote: Total 9618 (delta 13), reused 0 (delta 0), pack-reused 9587 Receiving objects: 100% (9618/9618), 113.08 MiB | 4.01 MiB/s, done. Resolving deltas: 100% (6853/6853), done. Checking connectivity... done. $ cd gkno_launcher $ ./gkno build

University of Utah gkno package

version: 2.53.5 date: May 2016

git commit: 394ac2de05295c659adfa19d5db4d15a2a6d43c0

Checking dependencies...done. Initializing component data...done. Building tools: bamaddrg...done. bamtools...done. bedtools...done. blast...done. bwa...done. freebayes...ERROR: See logs/build_freebayes.* files for more details. htslib...done. samtools...done. pindel...ERROR: See logs/build_pindel.* files for more details. libStatGen...done. bamUtil...done. fastQValidator...done. qplot...done. verifyBamID...done. mosaik...ERROR: See logs/build_mosaik.* files for more details. mutatrix...ERROR: See logs/build_mutatrix.* files for more details. picard...done. samblaster...done. seqan... snpEff...done. tabix...done. tangram...done. vcflib...ERROR: See logs/build_vcflib.* files for more details. vt...done. wham...ERROR: See logs/build_wham.* files for more details. Fetching default resources: tutorial: Downloading files... 100% Unpacking files...done.

WARNING: Not all tools were built successfully.

DETAILS: While building gkno, some of the tools failed to compile successfully. Please check the log files to determine the cause of the failures and rebuild. In the meantime, all other tools and pipelines not containing the failed tools can be used. The tools that failed to build are:

      pindel
      vcflib
      freebayes
      wham
      mosaik
      mutatrix

$

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/gkno/gkno_launcher/issues/67#issuecomment-220579284

bede commented 8 years ago

Thanks for the update Alastair, I'll have to try GKNO out on our Scientific Linux cluster. I'm pretty sure @ekg is aware of these issues and recommends GCC for the build on the Mac side, which IIRC can work, although I think I had to change more than just the CC and CXX environment variables to force the build to not to use clang/llvm.

Bede