Issues with Mosaik build and update

[CarlosBorroto]

Hi,

I ran into two issues with Mosaik. In my system, CentOS 6.4, Mosaik failed to build until I installed zlib-static and glibc-static. The issue in gkno is "./gkno build" didn't report this failed build. I only found the problem when I got this error while trying to run "./gkno run-test":

ERROR:   Missing executable files

DETAILS: The following executable files are not available, but are required:

         /home/cborroto/src/gkno_launcher/tools/MOSAIK/bin/MosaikBuild
         /home/cborroto/src/gkno_launcher/tools/MOSAIK/bin/MosaikJump
         /home/cborroto/src/gkno_launcher/tools/MOSAIK/bin/MosaikAligner

================================================================================================
  TERMINATED: Errors found in running gkno.  See specific error messages above for resolution.
================================================================================================

Also, when I tried "./gkno update", Mosaik would fail and point me to the .out and .err logs files. The problem is they were empty, no output whatsoever. Probably this is related to why a failed build is not detected.

After some pocking around I was able to fix the issue with update. I modified Mosaik's doUpdate in 'src/gkno/conf/gknoTools.py' by changing line:

    if not self.doPlatformExports()

to:

    self.doPlatformExports()
    if not self.BLD_PLATFORM:

I think the problem is at least in Python 2.6, the result from the value assigned in the last expression in a function is not returned, so this code from doPlatformExports does not returns TRUE ever:

  def doPlatformExports(self):
    pl='linux'
    if sys.platform == 'darwin':
      pl='macosx'
      if sys.maxsize > 2**32:
        pl='macosx64'
    self.BLD_PLATFORM = pl

I guess you could also make sure doPlatformExports does return TRUE once self.BLD_PLATFORM is properly set. Can't tell which would be better.

Thanks, Carlos

[pezmaster31]

Ah, I wasn't aware of those dependencies in Mosaik. Thanks for the heads-up.

And to the update issue, thanks for uncovering this missing return value bug. Can you confirm that simply returning True from doPlatformExports() fixes it? That approach just feels like better behavior to me.

[CarlosBorroto]

Yes, adding the return line does the trick too. In Mosaik's doPlatformExports:

    self.BLD_PLATFORM = pl
    return self.BLD_PLATFORM

And I'm guessing you probably would like to add the 'if not' check to doBuild. Is not clear to me when BLD_PLATFORM could map to a FALSE value, but I guess if there is a reason for the check in doUpdate the same applies to doBuild.

Regarding the issue with the failed build because of the missing dependencies. Please notice I'm not reporting 'gkno build' misses catching this. I think it will be extremely hard to catch every single missing dependency for all tools. Most people are used to look at the make error and figure what needs to be installed. The issue I'm reporting is, 'gkno build/update' does not detected when Mosaik fails to build.

If you want to reproduced this, make sure your system doesn't have zlib-devel or glibc-devel(in my case -static version of these packages) and run:

$ cd tools/MOSAIK/src
$ make clean
$ cd ../../../
$ ./gkno update

===============================
  Boston College gkno package

  version: 0.70
  date:    May 2013
===============================

Reading in command line arguments...done.

Updating gkno...done.
Initializing component data...done.
Checking tools:
  bamtools...done.
  freebayes...done.
  gatk...done.
  libStatGen...done.
  mosaik...done.
  ogap...done.
  picard...done.
  premo...done.
  samtools...done.
  tabix...done.
  tangram...done.
  vcflib...done.
Checking resources...
done.
$ ls tools/MOSAIK/bin/
MosaikJump
$ cd tools/MOSAIK/src/
$ make
Building MOSAIK for the following platform: linux
=========================================================
- Building in fann-2.1.0
make[1]: Nothing to be done for `all'.

- Building in CommonSource
- Building in DataStructures
make[2]: Nothing to be done for `all'.

- Building in ExternalReadFormats
make[2]: Nothing to be done for `all'.

- Building in MosaikReadFormat
make[2]: Nothing to be done for `all'.

- Building in PairwiseAlignment
make[2]: Nothing to be done for `all'.

- Building in Utilities
make[2]: Nothing to be done for `all'.

- Building in MosaikBuild
  * linking MosaikBuild
/usr/bin/ld: cannot find -lz
collect2: ld returned 1 exit status
make[1]: *** [MosaikBuild] Error 1

- Building in MosaikAligner
  * linking MosaikAligner
/usr/bin/ld: cannot find -lz
collect2: ld returned 1 exit status
make[1]: *** [MosaikAligner] Error 1

- Building in MosaikJump
  * linking MosaikJump
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/libpthread.a(libpthread.o): In function `sem_open':
(.text+0x764d): warning: the use of `mktemp' is dangerous, better use `mkstemp'

- Building in MosaikText
  * linking MosaikText
/usr/bin/ld: cannot find -lz
collect2: ld returned 1 exit status
make[1]: *** [MosaikText] Error 1

Hope this helps to clarify the issue I'm reporting.

Thanks, Carlos

[pezmaster31]

Please notice I'm not reporting 'gkno build' misses catching this... The issue I'm reporting is, 'gkno build/update' does not detected when Mosaik fails to build.

Sorry for not addressing this in my response, but yeah I caught that, and I'm already looking into it. :) Thanks for the extra info.

[pezmaster31]

Just pushed a fix that should address all you mentioned here:

• cleaned up the doPlatformExports() code.
• modified Mosaik's Makefile. It was swallowing errors internally, and just always returning zero. It should now fail so that gkno can catch & report the error during build/update.

As always, feel free to re-open if I've missed something.

• Derek

[pezmaster31]

Just a followup/clarification for anyone else that stumbles upon this thread - this fix didn't touch the static zlib / glibc dependency.

[bcantarel]

I am getting a similar error -- although not sure I follow how to fix it. I get this error after trying make in the src directory of Mosaik:

• Building in MosaikBuild
  • compiling BuildMain.cpp
  • compiling MosaikBuild.cpp
  • linking MosaikBuild /usr/bin/ld: cannot find -lz collect2: ld returned 1 exit status make[1]: * [MosaikBuild] Error 1 make: * [all] Error 1