make: mixed implicit and normal rules

[sjackman]

Courtney Hofman wrote...

Hi,

I installed Abyss 1.3.7 using homebrew and tried running abyss-pe on small mitogenome dataset. However I keep getting this error : /usr/local/bin/abyss-pe:613: *\ mixed implicit and normal rules. Stop.

I also tried the activity found at sjackman.github.io/abyss-activity using the run-abyss scripts (using the included data and following the directions) to see if it was just my data but I got the same error. I haven't seen anything in seq answers about this so I thought I would ask for any advice you might have to fix this problem.

Thanks, Courtney

[sjackman]

Hi, Courtney.

What version of Mac OS are your running?

Please run and report the output of the following commands:

Cheers, Shaun

which -a make
make --version
/usr/bin/make --version
/usr/local/bin/make --version
system_profiler SPSoftwareDataType
which -a xcodebuild
xcodebuild -version
/usr/bin/xcodebuild -version
/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version
brew doctor

[chofman]

Hi Shaun,

Thanks for getting back to me so quickly, below is the output for those commands:

Courtney@courtneys-air:~top|⇒  which -a make
/usr/bin/make
/usr/bin/make
Courtney@courtneys-air:~top|⇒  make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0
Courtney@courtneys-air:~top|⇒  /usr/bin/make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0
Courtney@courtneys-air:~top|⇒  /usr/local/bin/make --version
zsh: no such file or directory: /usr/local/bin/make
Courtney@courtneys-air:~top|⇒  system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: OS X 10.9.1 (13B42)
      Kernel Version: Darwin 13.0.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: Courtney’s MacBook Air
      User Name: Courtney Hofman (Courtney)
      Secure Virtual Memory: Enabled
      Time since boot: 16 days 20:44

Courtney@courtneys-air:~top|⇒  which -a xcodebuild
/usr/bin/xcodebuild
/usr/bin/xcodebuild
Courtney@courtneys-air:~top|⇒  xcodebuild -version
Xcode 5.0.2
Build version 5A3005
Courtney@courtneys-air:~top|⇒  /usr/bin/xcodebuild -version
Xcode 5.0.2
Build version 5A3005
Courtney@courtneys-air:~top|⇒  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version
Xcode 5.0.2
Build version 5A3005

Warning: Some directories in /usr/local/share/man aren't writable.
This can happen if you "sudo make install" software that isn't managed
by Homebrew. If a brew tries to add locale information to one of these
directories, then the install will fail during the link step.
You should probably `chown` them:

    /usr/local/share/man/mann

Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected dylibs:
    /usr/local/lib/libtcl8.6.dylib
    /usr/local/lib/libtk8.6.dylib

Warning: Unbrewed .pc files were found in /usr/local/lib/pkgconfig.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected .pc files:
    /usr/local/lib/pkgconfig/tcl.pc
    /usr/local/lib/pkgconfig/tk.pc

Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected static libraries:
    /usr/local/lib/libtclstub8.6.a
    /usr/local/lib/libtkstub8.6.a

Warning: Some installed formula are missing dependencies.
You should `brew install` the missing dependencies:

    brew install openssl

Run `brew missing` for more details.

Warning: /usr/bin occurs before /usr/local/bin
This means that system-provided programs will be used instead of those
provided by Homebrew. The following tools exist at both paths:

    2to3
    2to3-2.7
    config_data
    crc32
    easy_install
    easy_install-2.7
    git
    git-cvsserver
    git-receive-pack
    git-shell
    git-upload-archive
    git-upload-pack
    idle
    idle2.7
    instmodsh
    prove
    ptar
    ptardiff
    ptargrep
    pydoc
    pydoc2.7
    python
    python-config
    python2.7
    python2.7-config
    pythonw
    pythonw2.7
    samtools
    samtools.pl
    shasum
    smtpd.py
    smtpd2.7.py
    zipdetails

Consider setting your PATH so that /usr/local/bin
occurs before /usr/bin. Here is a one-liner:
    echo export PATH='/usr/local/bin:$PATH' >> ~/.bash_profile

Thanks again for your help, I probably screwed up something during prior installations.

Courtney

[sjackman]

@traymond Add Tony

[sjackman]

Your version of make doesn't seem to be the issue. I use the same GNU Make 3.81 on Mac OS.

Just a sanity check, but can you post the complete output of these commands:

wget http://www.bcgsc.ca/platform/bioinfo/software/abyss/releases/1.3.4/test-data.tar.gz
tar xzvf test-data.tar.gz
abyss-pe k=25 name=test \
    in='test-data/reads1.fastq test-data/reads2.fastq'

[chofman]

Here is that output:

Courtney@courtneys-air:~top|⇒  tar xzvf test-data.tar.gz 
x test-data/
x test-data/reads1.fastq
x test-data/reads2.fastq
Courtney@courtneys-air:~top|⇒  abyss-pe k=25 name=test in='test-data/reads1.fast test-    data/reads2.fasta'
/usr/local/bin/abyss-pe:613: *** mixed implicit and normal rules.  Stop.

[sjackman]

Please report the output of

echo x"$ref"y
env |grep ref
set |grep ref
unset ref
abyss-pe k=25 name=test in='test-data/reads1.fastq test-data/reads2.fastq'

[chofman]

Here is the output Courtney@courtneys-air:~top|⇒ echo x"$ref"y x/Volumes/Courtney's HardDrive/Catalina Squirrel/abyss/chr3.fay Courtney@courtneys-air:~top|⇒ env|grep ref ref=/Volumes/Courtney's HardDrive/Catalina Squirrel/abyss/chr3.fa Courtney@courtneys-air:~top|⇒ set |grep ref Binary file (standard input) matches Courtney@courtneys-air:~top|⇒ unset ref Courtney@courtneys-air:~top|⇒ abyss-pe k=25 name=test in='test-data/reads1.fastq test-data/reads2.fastq' Abyss ran and below is the abyss-fac info abyss-fac test-unitigs.fa test-contigs.fa test-scaffolds.fa |tee test-stats.tab n n:500 n:N50 min N80 N50 N20 E-size max sum name 817 108 38 509 642 905 1323 1008 2260 93364 test-unitigs.fa 500 78 19 500 1447 3101 5099 3676 10702 172641 test-contigs.fa 425 29 7 578 4828 9965 17044 11621 27989 179916 test- scaffolds.fa ln -sf test-stats.tab test-stats tr '\t' , <test-stats.tab >test-stats.csv abyss-tabtomd test-stats.tab >test-stats.md

[sjackman]

Hi, Courtney. The problem is two-fold. One, that the environment variable ref is set, which abyss-pe is interpreting as a parameter intended for it. And two, that the file name pointed to by $ref has a space in it, or maybe it's the apostrophe causing trouble. Although it is a bug, ABySS likely won't be the only software to barf like this. To make your life easier, you may want to consider removing special characters from your volume and folder names.

@traymond Could you look into this? We should add a warning to the documentation that environment variables with the same names as abyss-pe parameters are used. We should test whether special characters like spaces break abyss-pe. If it does break it, and it can be fixed, great, otherwise we should document that file names shouldn't contain special characters (probably any character interpreted by the shell).

[sjackman]

@chofman Run unset ref before assembling your mitogenome, and ABySS should work. If not, get back to us with the new error message.

[traymond]

I've always considered the fact that you can pass abyss-pe parameters by exporting them as a feature. As far as I can tell, the only difference between environment variables and make variables is that environment variables can be overwritten by a normal assignment. I could use this property to try and detect when we are using an environment variable, but I'm pretty sure the abyss-pe script will get quite convoluted quite quickly if I do this for all the variables.

I'll look into returning an error if there's a space in the variable.

[sjackman]

I agree that the ability to use environment variables is a feature of abyss-pe. It's just surprising if it's not expected, like in this case. I don't think we need to add code to abyss-pe, necessarily, but this behaviour should be documented.

[traymond]

Ah, yeah. A list of variable names that must be unset before running abyss-pe would be good.

[sjackman]

A command abyss-pe env that reports the value of all the abyss-pe configuration variables would be super useful for troubleshooting.

[ghost]

How about something like f2a2ce941cb3b6458d6d4c8b6df17c386183a572 ? (Note that $HOME, $HOSTNAME are for testing purpose)

[ghost]

sorry, above link doesn't work. New code is in -dev, 'check_env' branch.

[traymond]

When you want to specify a commit in another repo, you need to specify the repo. Like this: bcgsc/abyss-dev@f2a2ce941cb3b6458d6d4c8b6df17c386183a572

[sjackman]

I like it. That's a nice diagnostic tool, Mimi. My thought was an env command of abyss-pe that reports the value of each variable after parsing the environment and command line. I could see a use for both tools.

[ghost]

That's a great idea, @sjackman. I will try that way too, sometime during this week. Thanks!

[ghost]

I got it! :) bcgsc/abyss-dev@298002aeff2c77eca17b289d1424074f5541ad4d now runs like this:

abyss-pe env abyss-pe env k=44 name=test in='some-files' abyss-pe k=44 name=test in='some-files' env

to report ALL defined/set abyss variables. Also, try after setting some environment variables, too!

[ghost]

moved to bcgsc/abyss-dev@65b7eed3dc2ae06a68cce33236629ea753f34d0a

[traymond]

This has been released in 1.5.0.