gkno build troubleshooting

[S-ES]

Hi Guys, I just started to build gkno on my personal MacBook, the building process seemed ok, however some tools were not built: verifyBamID, musket, rufus, jellyfish, mutatrix.. log files :

• build_jellyfish.err: fatal: destination path 'yaggo' already exists and is not an empty directory.
• build_musket.err: kmer.cpp: In member function ‘Kmer Kmer::set_base(uint8_t, int)’: kmer.cpp:133: warning: comparison is always true due to limited range of data type In file included from thread.h:7, from option.h:21, from main.cpp:1: barrier.h:19: error: ‘pthread_barrier_t’ does not name a type barrier.h: In constructor ‘MyBarrier::MyBarrier(int)’: barrier.h:10: error: ‘barrier’ was not declared in this scope barrier.h:10: error: ‘pthread_barrier_init’ was not declared in this scope barrier.h: In destructor ‘MyBarrier::~MyBarrier()’: barrier.h:13: error: ‘barrier’ was not declared in this scope barrier.h:13: error: ‘pthread_barrier_destroy’ was not declared in this scope barrier.h: In member function ‘void MyBarrier::wait()’: barrier.h:16: error: ‘barrier’ was not declared in this scope barrier.h:16: error: ‘pthread_barrier_wait’ was not declared in this scope main.cpp: In function ‘void ParaKmerEC(ProgramOptions&)’: main.cpp:650: warning: iteration variable ‘readIdx’ is unsigned make: * [main.o] Error 1 make: * Waiting for unfinished jobs.... In file included from thread.h:7, from thread.cpp:1: barrier.h:19: error: ‘pthread_barrier_t’ does not name a type barrier.h: In constructor ‘MyBarrier::MyBarrier(int)’: barrier.h:10: error: ‘barrier’ was not declared in this scope barrier.h:10: error: ‘pthread_barrier_init’ was not declared in this scope barrier.h: In destructor ‘MyBarrier::~MyBarrier()’: barrier.h:13: error: ‘barrier’ was not declared in this scope barrier.h:13: error: ‘pthread_barrier_destroy’ was not declared in this scope barrier.h: In member function ‘void MyBarrier::wait()’: barrier.h:16: error: ‘barrier’ was not declared in this scope barrier.h:16: error: ‘pthread_barrier_wait’ was not declared in this scope make: *\ [thread.o] Error 1

-build_mutatrix.err: In file included from Fasta.h:19, from Fasta.cpp:9: LargeFileSupport.h:12: error: ‘__off64_t’ does not name a type make[1]: * [Fasta.o] Error 1 make: * [fastahack/Fasta.o] Error 2 make: *\ Waiting for unfinished jobs.... LeftAlign.cpp:679:26: warning: '&&' within '||' [-Wlogical-op-parentheses] && (indel.insertion && indel.position == referenceSequence.size()

LeftAlign.cpp:679:26: note: place parentheses around the '&&' expression to silence this warning
            && (indel.insertion && indel.position == referenceSequence.size()
                                ^
                (                                                            )
1 warning generated.
clang: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated
ld: warning: option -s is obsolete and being ignored
i686-apple-darwin11-llvm-g++-4.2: -lm: linker input file unused because linking not done
i686-apple-darwin11-llvm-g++-4.2: -ltabix: linker input file unused because linking not done
i686-apple-darwin11-llvm-g++-4.2: -lz: linker input file unused because linking not done
i686-apple-darwin11-llvm-g++-4.2: -lm: linker input file unused because linking not done
i686-apple-darwin11-llvm-g++-4.2: -ltabix: linker input file unused because linking not done
i686-apple-darwin11-llvm-g++-4.2: -lz: linker input file unused because linking not done
i686-apple-darwin11-llvm-g++-4.2: -lm: linker input file unused because linking not done

-build_rufus.err:
cc1plus: error: unrecognized command line option "-std=gnu++0x"
cc1plus: error: unrecognized command line option "-std=gnu++0x"
make: **\* [bin/ModelDist2] Error 1
make: **\* Waiting for unfinished jobs....
make: **\* [bin/Overlap19] Error 1

-build_VerifyBamID
Main.cpp:23:20: error: values.h: No such file or directory
make[1]: **\* [../obj/Main.o] Error 1
make: **\* [src] Error 2

Do you have any idea why ? Has anyone encountered these issue before ?

Best regards

Kiz

[ekg]

Looks like this is the problem:

LargeFileSupport.h:12: error: ‘__off64_t’ does not name a type

I believe there is a workaround, but I can't remember it off the top of my head.

On Thu, May 15, 2014 at 5:27 PM, Kizuna777 notifications@github.com wrote:

Hi Guys, I just started to build gkno on my personal MacBook, the building process seemed ok, however some tools were not built: verifyBamID, musket, rufus, jellyfish, mutatrix.. log files :

-

build_jellyfish.err: fatal: destination path 'yaggo' already exists and is not an empty directory.

build_musket.err: kmer.cpp: In member function ‘Kmer Kmer::set_base(uint8_t, int)’: kmer.cpp:133: warning: comparison is always true due to limited range of data type In file included from thread.h:7, from option.h:21, from main.cpp:1: barrier.h:19: error: ‘pthread_barrier_t’ does not name a type barrier.h: In constructor ‘MyBarrier::MyBarrier(int)’: barrier.h:10: error: ‘barrier’ was not declared in this scope barrier.h:10: error: ‘pthread_barrier_init’ was not declared in this scope barrier.h: In destructor ‘MyBarrier::~MyBarrier()’: barrier.h:13: error: ‘barrier’ was not declared in this scope barrier.h:13: error: ‘pthread_barrier_destroy’ was not declared in this scope barrier.h: In member function ‘void MyBarrier::wait()’: barrier.h:16: error: ‘barrier’ was not declared in this scope barrier.h:16: error: ‘pthread_barrier_wait’ was not declared in this scope main.cpp: In function ‘void ParaKmerEC(ProgramOptions&)’: main.cpp:650: warning: iteration variable ‘readIdx’ is unsigned make: * [main.o] Error 1 make: * Waiting for unfinished jobs.... In file included from thread.h:7, from thread.cpp:1: barrier.h:19: error: ‘pthread_barrier_t’ does not name a type barrier.h: In constructor ‘MyBarrier::MyBarrier(int)’: barrier.h:10: error: ‘barrier’ was not declared in this scope barrier.h:10: error: ‘pthread_barrier_init’ was not declared in this scope barrier.h: In destructor ‘MyBarrier::~MyBarrier()’: barrier.h:13: error: ‘barrier’ was not declared in this scope barrier.h:13: error: ‘pthread_barrier_destroy’ was not declared in this scope barrier.h: In member function ‘void MyBarrier::wait()’: barrier.h:16: error: ‘barrier’ was not declared in this scope barrier.h:16: error: ‘pthread_barrier_wait’ was not declared in this scope make: *\ [thread.o] Error 1

-build_mutatrix.err: In file included from Fasta.h:19, from Fasta.cpp:9: LargeFileSupport.h:12: error: ‘__off64_t’ does not name a type make[1]: * [Fasta.o] Error 1 make: * [fastahack/Fasta.o] Error 2 make: *\ Waiting for unfinished jobs.... LeftAlign.cpp:679:26: warning: '&&' within '||' [-Wlogical-op-parentheses] && (indel.insertion && indel.position == referenceSequence.size()

LeftAlign.cpp:679:26: note: place parentheses around the '&&' expression
to silence this warning
&& (indel.insertion && indel.position == referenceSequence.size()
^
( )
1 warning generated.
clang: warning: treating 'c' input as 'c++' when in C++ mode, this
behavior is deprecated
ld: warning: option -s is obsolete and being ignored
i686-apple-darwin11-llvm-g++-4.2: -lm: linker input file unused because
linking not done
i686-apple-darwin11-llvm-g++-4.2: -ltabix: linker input file unused
because linking not done
i686-apple-darwin11-llvm-g++-4.2: -lz: linker input file unused because
linking not done
i686-apple-darwin11-llvm-g++-4.2: -lm: linker input file unused because
linking not done
i686-apple-darwin11-llvm-g++-4.2: -ltabix: linker input file unused
because linking not done
i686-apple-darwin11-llvm-g++-4.2: -lz: linker input file unused because
linking not done
i686-apple-darwin11-llvm-g++-4.2: -lm: linker input file unused because
linking not done

-build_rufus.err:
cc1plus: error: unrecognized command line option "-std=gnu++0x"
cc1plus: error: unrecognized command line option "-std=gnu++0x"
make: *** [bin/ModelDist2] Error 1
make: *** Waiting for unfinished jobs....
make: *** [bin/Overlap19] Error 1

-build_VerifyBamID
Main.cpp:23:20: error: values.h: No such file or directory
make[1]: *** [../obj/Main.o] Error 1
make: *** [src] Error 2

Do you have any idea why ? Has anyone encountered these issue before ?

Best regards

Kiz

—
Reply to this email directly or view it on GitHub<https://github.com/gkno/gkno_launcher/issues/10>
.

[alexpreynolds]

Adding the following macro in the LargeFileSupport header may help:

#ifdef __APPLE__
#  define off64_t off_t
#endif

[AlistairNWard]

We have solutions to a number of the tools, but I'm waiting on one more update to one of the tools. Once this is complete (should be done today), I'll post a list of steps that should get most of the tools up and running. Unfortunately there are a couple of tools that we are a problem for compilation on a Mac and it may take a little more time to get the last few to compile. Thank you for your patience.

On Fri, May 16, 2014 at 6:51 PM, Kizuna777 notifications@github.com wrote:

sorry guys but I did not get it right.. may you please explain it to more in details ..

— Reply to this email directly or view it on GitHubhttps://github.com/gkno/gkno_launcher/issues/10#issuecomment-43387632 .

[AlistairNWard]

Ok, so here is a list of steps that can be included to help some of the tools. verifyBamID has been updated to remove the inclusion of values.h, so that should now work. If you have homebrew, you can try using the compiler that comes with that (that will at least solve the problems with RUFUS). If you don't have homebrew, you can install it using:

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

Then install gcc:

brew install gcc

You will need to add soft links to generate the executables for gcc and g++.

cd /usr/local/bin ln -s g++ g++-4.8 ln -s gcc gcc-4.8

Then modify your path, so that when g++ is called, it looks in the directory with homebrew installations first and modify the CXX environment variable.

PATH=/usr/local/bin:$PATH export CXX=g++

Now navigate back to your gkno_launcher directory (with a fresh clone) and try to build. Let me know if this solves any of the problems.

You can log out of your shell and log back in to reset the PATH and CXX variables.

[S-ES]

Hi ! I will try it and get back to you very soon :) Thx

[AlistairNWard]

I have made a few more updates to things and also the mutatrix and scissors codes have been updated to help with compilation on a Mac. Please try a fresh clone and (hopefully) you'll get everything except musket and Jellyfish to compile. I'm still looking into these last two and hopefully we'll get those Mac compatible soon as well.

[S-ES]

After installing gcc via homebrew and downloading a new clone, all the tools were built except jellyfish and Musket.. However I have noticed two things:

1- the first time I build gkno i have this error message:

jellyfish...Traceback (most recent call last): File "/Users/apple/gkno_launcher/src/gkno.py", line 484, in main() File "/Users/apple/gkno_launcher/src/gkno.py", line 281, in main success = admin.run(sys.argv) File "/Users/apple/gkno_launcher/src/gkno/adminUtils.py", line 84, in run if self.mode == "build" : success = self.build() File "/Users/apple/gkno_launcher/src/gkno/adminUtils.py", line 153, in build if self.buildTool(tool): File "/Users/apple/gkno_launcher/src/gkno/adminUtils.py", line 539, in buildTool success = tool.build(out, err) File "/Users/apple/gkno_launcher/src/gkno/conf/gknoTools.py", line 73, in build return self.doBuild() File "/Users/apple/gkno_launcher/src/gkno/conf/gknoTools.py", line 333, in doBuild if not self.runCommand("autoreconf -i") : return False File "/Users/apple/gkno_launcher/src/gkno/conf/gknoTools.py", line 90, in runCommand p = subprocess.Popen(command.split(), env=self.environ, stdout=self.out, stderr=self.err) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 709, in init errread, errwrite) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1326, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

This error message disappear during the second building, and the process goes "OK" (see attached plz) .

2- the second issue is when I run ./gkno run-test, after a "correct build", this is what I am having:

Reading in command line arguments...done. Checking instance information...done. Assigning command line arguments to graph nodes...done. Checking for commands to execute at command line...done.

Workflow: merge-reference (concatenate-files): Join multiple files build-reference (mosaik-build-reference): Build the Mosaik reference build-jump-database (mosaik-jump): Generate the jump database for a Mosaik reference build-read-archive (mosaik-build-fastq): Build the Mosaik read archive align (mosaik-aligner-special): Pairwise alignment of a read archive with additional 'special' reference sequences. The special sequences must all have a common prefix and alignment to them will be shown in the ZA tags. No primary alignments to the 'special' sequences will occur. sort-primary-bam (bamtools-sort): Sort a BAM file recalibrate-bq (recab): Recalibrate base qualities in a BAM file (University of Michigan). mark-duplicates (dedup): Mark duplicate reads in a BAM file (University of Michigan). index-final-bam (bamtools-index): Index a BAM file. variant-call (freebayes): Bayesian variant and haplotype calling filter-variants (vcflib-filter): Filter vcf files on a variety of parameters.

ERROR: A required command line argument is missing.

DETAILS: The task 'variant-call' requires the argument '--region (-r)' to be set, but it has not been specified on the command line. This argument cannot be set using a pipeline argument and consequently must be set using the syntax:

     ./gkno pipe <pipeline name> --variant-call [--region <value>] [options]

     This argument is described as the following: <chrom>:<start_position>..<end_position>.
     Limit analysis to the specified region, 0-base coordinates, end_position not included
     (same as BED format).

     It is recommended that the pipeline configuration file be modified to ensure that all
     arguments required by the pipeline have a command line argument defined in the pipeline
     configuration file. Please see the documentation for further information on how this
     can be accomplished.

TERMINATED: Errors found in running gkno. See specific error messages above for resolution.

I hope this helps !

Thx

Kiz

[AlistairNWard]

I have seen that crash on the first build myself and am trying to get to the bottom of it. It only seems to occur on Macs, and even then doesn't seem to be consistent. I can replicate it, so will get rid of it. Musket uses methods that are known to not work on Macs. It is possible to solve this, so I may have to go into the code and include a workaround to get it to work. I will look into all the Jellyfish problems first, though and try and get that going.

I apologise about the run-test pipeline. I had made an update and forgotten to update the configuration files. If you run gkno update, you should pull in updated files and this should work. Please let us know if there are any additional tools/pipelines that you would like to see included in gkno and thanks again for your comments and bug reports.

[S-ES]

Thx for your prompt response and for your efforts for making these tools available for rookies in informatics (my case)..

I will gladly propose additional tools/pipelines if needed !

P.S: I will post this fruitful conversation on bio-stars, so others also can benefit from this update..

[AlistairNWard]

I'm glad this helped and I hope you find the tool useful. We are currently actively working on a few pipelines that should be very useful, so keep your eye on our twitter feed for updates (@gknoProject).

On Sat, May 24, 2014 at 10:23 AM, Kizuna777 notifications@github.comwrote:

Thx for your prompt response and for your efforts for making these tools available for rookies in informatics (my case)..

I will gladly propose additional tools/pipelines if needed !

P.S: I will post this fruitful conversation on bio-stars, so others also can benefit from this update..

— Reply to this email directly or view it on GitHubhttps://github.com/gkno/gkno_launcher/issues/10#issuecomment-44088672 .

[AlistairNWard]

Sorry for the piecemeal set of solutions to this issue. When I have everything sorted, I'll include a Mac compilation tutorial on the gkno website. The crash that you see on the first attempt to build gkno is because automake isn't available on your system. Try typing 'autoreconf' and I suspect that you'll see a 'command not found' message. If you type 'brew install automake', this will stop that crash.

[S-ES]

Indeed, after installing automake, the crash stopped !

Regarding some interesting tools that might be added to gkno; I would propose to add a cnv detection tool.. In fact, I have checked Tangram, but if I understood well, this tool currently detects MEI and does not yet include CNVs. I would suggest CoNIFER (http://conifer.sourceforge.net/) as it is currently used by many teams and as it was shown a bit more sensitive than the other tools.. (Detection of Clinically Relevant Copy Number Variants with Whole-Exome Sequencing, Joep de Ligt, et al. 2013). CoNIFER is mainly used for WES, however, some are saying that it can be also customized for TES (targueted Exome Sequencing).. if this tool can be included in gkno with the possiblity of undergoing TES and WES, this option might be a plus for GKNO..

I hope this might be useful.

Best !

Kiz

[AlistairNWard]

Ok, so I am going to close this issue and leave the tool Musket as not available on a Mac. There is a comprehensive set of steps to follow for getting this to work on Mac OSX 10.9. In order to get Jellyfish to compile, all these steps are required. So, first of all a number of packages are required. These are downloaded using homebrew, which can be downloaded using the first of the commands listed below. You should be able to cut and paste all of the commands.

Get homebrew:

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

Next, install all the required packages:

brew install gcc brew install automake brew install valgrind brew install pkg-config

Now we need to set some links:

cd /usr/local/bin ln -s g++ g++-4.8 ln -s gcc gcc-4.8

cd /usr/local/Cellar/libtool/2.4.2/bin ln -s glibtool libtool ln -s glibtoolize libtoolize

Then modify the environment variable for CXX:

export CXX=g++

Finally, modify the path to see the correct versions of gcc and libtool.

PATH=/usr/local/bin:/usr/local/Cellar/libtool/2.4.2/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

Now navigate back to your gkno_launcher directory (with a fresh clone) and try to build. You can log out of your shell and log back in to reset the PATH and CXX variables.