search

merenlab / anvio

An analysis and visualization platform for 'omics data
http://merenlab.org/software/anvio
GNU General Public License v3.0
439 stars 145 forks source link

anvi'o installs on Intel Core i7 MBP (2011) but not Intel quad-core Xeon (2009 Mac Pro) #362

Closed jbadomics closed 8 years ago

jbadomics commented 8 years ago

I am using the anvi'o Mac installer, and I kept getting the same error (entire log posted below) on the lab machine which is a 2009 Mac Pro with 2 quad-core 2.66 GHz Xeon processors running OS X 10.11.4. On my personal laptop (2011 Macbook Pro, 2.7 GHz i7) it installs just fine - also running 10.11.4. Aside from the hardware difference the software configuration is essentially identical.

On the 2009 Mac Pro:

$ sysctl -a | grep machdep.cpu
machdep.cpu.max_basic: 11
machdep.cpu.max_ext: 2147483656
machdep.cpu.vendor: GenuineIntel
machdep.cpu.brand_string: Intel(R) Xeon(R) CPU           W3520  @ 2.67GHz
machdep.cpu.family: 6
machdep.cpu.model: 26
machdep.cpu.extmodel: 1
machdep.cpu.extfamily: 0
machdep.cpu.stepping: 5
machdep.cpu.feature_bits: 44160500475100159
machdep.cpu.extfeature_bits: 4967106816
machdep.cpu.signature: 67237
machdep.cpu.brand: 0
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 DTES64 MON DSCPL VMX EST TM2 SSSE3 CX16 TPR PDCM SSE4.1 SSE4.2 POPCNT
machdep.cpu.extfeatures: SYSCALL XD EM64T LAHF RDTSCP TSCI
machdep.cpu.logical_per_package: 16
machdep.cpu.cores_per_package: 8
machdep.cpu.microcode_version: 17
machdep.cpu.processor_flag: 1
machdep.cpu.mwait.linesize_min: 64
machdep.cpu.mwait.linesize_max: 64
machdep.cpu.mwait.extensions: 3
machdep.cpu.mwait.sub_Cstates: 4384
machdep.cpu.thermal.sensor: 1
machdep.cpu.thermal.dynamic_acceleration: 1
machdep.cpu.thermal.invariant_APIC_timer: 0
machdep.cpu.thermal.thresholds: 2
machdep.cpu.thermal.ACNT_MCNT: 1
machdep.cpu.thermal.core_power_limits: 0
machdep.cpu.thermal.fine_grain_clock_mod: 0
machdep.cpu.thermal.package_thermal_intr: 0
machdep.cpu.thermal.hardware_feedback: 0
machdep.cpu.thermal.energy_policy: 0
machdep.cpu.arch_perf.version: 3
machdep.cpu.arch_perf.number: 4
machdep.cpu.arch_perf.width: 48
machdep.cpu.arch_perf.events_number: 7
machdep.cpu.arch_perf.events: 68
machdep.cpu.arch_perf.fixed_number: 3
machdep.cpu.arch_perf.fixed_width: 48
machdep.cpu.cache.linesize: 64
machdep.cpu.cache.L2_associativity: 8
machdep.cpu.cache.size: 256
machdep.cpu.tlb.inst.small: 64
machdep.cpu.tlb.inst.large: 7
machdep.cpu.tlb.data.small: 64
machdep.cpu.tlb.data.large: 32
machdep.cpu.tlb.shared: 512
machdep.cpu.address_bits.physical: 36
machdep.cpu.address_bits.virtual: 48
machdep.cpu.core_count: 4
machdep.cpu.thread_count: 8
machdep.cpu.flex_ratio.desired: 63
machdep.cpu.flex_ratio.min: 12
machdep.cpu.flex_ratio.max: 20
machdep.cpu.tsc_ccc.numerator: 0
machdep.cpu.tsc_ccc.denominator: 0

On the 2011 Macbook Pro:

$ sysctl -a | grep machdep.cpu
machdep.cpu.max_basic: 13
machdep.cpu.max_ext: 2147483656
machdep.cpu.vendor: GenuineIntel
machdep.cpu.brand_string: Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
machdep.cpu.family: 6
machdep.cpu.model: 42
machdep.cpu.extmodel: 2
machdep.cpu.extfamily: 0
machdep.cpu.stepping: 7
machdep.cpu.feature_bits: 2286390448420027391
machdep.cpu.extfeature_bits: 4967106816
machdep.cpu.signature: 132775
machdep.cpu.brand: 0
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC POPCNT AES PCID XSAVE OSXSAVE TSCTMR AVX1.0
machdep.cpu.extfeatures: SYSCALL XD EM64T LAHF RDTSCP TSCI
machdep.cpu.logical_per_package: 16
machdep.cpu.cores_per_package: 8
machdep.cpu.microcode_version: 40
machdep.cpu.processor_flag: 4
machdep.cpu.mwait.linesize_min: 64
machdep.cpu.mwait.linesize_max: 64
machdep.cpu.mwait.extensions: 3
machdep.cpu.mwait.sub_Cstates: 135456
machdep.cpu.thermal.sensor: 1
machdep.cpu.thermal.dynamic_acceleration: 1
machdep.cpu.thermal.invariant_APIC_timer: 1
machdep.cpu.thermal.thresholds: 2
machdep.cpu.thermal.ACNT_MCNT: 1
machdep.cpu.thermal.core_power_limits: 1
machdep.cpu.thermal.fine_grain_clock_mod: 1
machdep.cpu.thermal.package_thermal_intr: 1
machdep.cpu.thermal.hardware_feedback: 0
machdep.cpu.thermal.energy_policy: 1
machdep.cpu.xsave.extended_state: 7 832 832 0
machdep.cpu.xsave.extended_state1: 1 0 0 0
machdep.cpu.arch_perf.version: 3
machdep.cpu.arch_perf.number: 4
machdep.cpu.arch_perf.width: 48
machdep.cpu.arch_perf.events_number: 7
machdep.cpu.arch_perf.events: 0
machdep.cpu.arch_perf.fixed_number: 3
machdep.cpu.arch_perf.fixed_width: 48
machdep.cpu.cache.linesize: 64
machdep.cpu.cache.L2_associativity: 8
machdep.cpu.cache.size: 256
machdep.cpu.tlb.inst.small: 64
machdep.cpu.tlb.inst.large: 8
machdep.cpu.tlb.data.small: 64
machdep.cpu.tlb.data.large: 32
machdep.cpu.tlb.shared: 512
machdep.cpu.address_bits.physical: 36
machdep.cpu.address_bits.virtual: 48
machdep.cpu.core_count: 2
machdep.cpu.thread_count: 4
machdep.cpu.tsc_ccc.numerator: 0
machdep.cpu.tsc_ccc.denominator: 0

Here's the full error log:

16:40 bondlab [~] $ source /Applications/Anvio.app/Contents/Resources/Scripts/activate.sh

16:41 bondlab [~] $ anvi-self-test

#
#
# Creating the output directory ...
#
#

#
#
# Anvo'o version ...
#
#

Anvi'o version ...............................: 1.2.2
Contigs DB version ...........................: 3
Profile DB version ...........................: 6
Samples information DB version ...............: 2
Auxiliary HDF5 DB version ....................: 1

#
#
# Initializing raw BAM files ...
#
#

Sorted BAM File ..............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-6M            
Indexed BAM File .............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-6M.bam        

Sorted BAM File ..............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-7M            
Indexed BAM File .............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-7M.bam        

Sorted BAM File ..............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-9M            
Indexed BAM File .............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-9M.bam        

#
#
# Generating an EMPTY contigs database ...
#
#

Contigs database .............................: A new database, test-output/CONTIGS.db, has been created.
Number of contigs ............................: 6
Number of splits .............................: 60
Total number of nucleotides ..................: 57,030
Split length .................................: 1,000

#
#
# Populating the genes tables in the database using 'myrast_cmdline' parser ...
#
#

Percent of contigs annotated .................: 50.0%

#
#
# Populating search tables in the latest contigs database using default HMM profiles ...
#
#

HMM profiles .................................: 4 sources have been loaded: Alneberg_et_al (34 genes), Dupont_et_al (111 genes), Campbell_et_al (139 genes), Creevey_et_al (40 genes)
FASTA for contigs ............................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpVHCGmu/contigs.fa                       

Finding ORFs in contigs
===============================================
Genes ........................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmp9h2uKI/contigs.genes
Proteins .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmp9h2uKI/contigs.proteins
Log file .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmp9h2uKI/00_log.txt

HMM Profiling for Alneberg_et_al
===============================================
Reference ....................................: Alneberg et al, http://www.nature.com/nmeth/journal/v11/n11/full/nmeth.3103.html
Pfam model ...................................: /Applications/Anvio.app/Contents/Resources/base/anvio/data/hmm/Alneberg_et_al/genes.hmm.gz
Number of genes ..............................: 34
Temporary work dir ...........................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmp1xwAa7
HMM scan output ..............................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmp1xwAa7/hmm.output
HMM scan hits ................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmp1xwAa7/hmm.hits
Log file .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmp1xwAa7/00_log.txt
Number of raw hits ...........................: 0                                                                                           

HMM Profiling for Dupont_et_al
===============================================
Reference ....................................: Dupont et al, http://www.nature.com/ismej/journal/v6/n6/full/ismej2011189a.html
Pfam model ...................................: /Applications/Anvio.app/Contents/Resources/base/anvio/data/hmm/Dupont_et_al/genes.hmm.gz
Number of genes ..............................: 111
Temporary work dir ...........................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpOqJSMl
HMM scan output ..............................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpOqJSMl/hmm.output
HMM scan hits ................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpOqJSMl/hmm.hits
Log file .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpOqJSMl/00_log.txt
Number of raw hits ...........................: 2                                                                                           

HMM Profiling for Campbell_et_al
===============================================
Reference ....................................: Campbell et al, http://www.pnas.org/content/110/14/5540.short
Pfam model ...................................: /Applications/Anvio.app/Contents/Resources/base/anvio/data/hmm/Campbell_et_al/genes.hmm.gz
Number of genes ..............................: 139
Temporary work dir ...........................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmplwjaZF
HMM scan output ..............................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmplwjaZF/hmm.output
HMM scan hits ................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmplwjaZF/hmm.hits
Log file .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmplwjaZF/00_log.txt
Number of raw hits ...........................: 2                                                                                           

HMM Profiling for Creevey_et_al
===============================================
Reference ....................................: Creevey et al, http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0022099
Pfam model ...................................: /Applications/Anvio.app/Contents/Resources/base/anvio/data/hmm/Creevey_et_al/genes.hmm.gz
Number of genes ..............................: 40
Temporary work dir ...........................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpRQU3YB
HMM scan output ..............................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpRQU3YB/hmm.output
HMM scan hits ................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpRQU3YB/hmm.hits
Log file .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpRQU3YB/00_log.txt
Number of raw hits ...........................: 0                                                                                           

#
#
# Populating search tables in the latest contigs database using a mock HMM collection from an external directory ...
#
#

HMM profiles .................................: 1 source been loaded: external_hmm_profile (2 genes)
FASTA for contigs ............................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmp8yokYp/contigs.fa                       

Finding ORFs in contigs
===============================================
Genes ........................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpw7mCD5/contigs.genes
Proteins .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpw7mCD5/contigs.proteins
Log file .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpw7mCD5/00_log.txt

HMM Profiling for external_hmm_profile
===============================================
Reference ....................................: Anvio et al, http://merenlab.org/projects/anvio
Pfam model ...................................: external_hmm_profile/genes.hmm.gz
Number of genes ..............................: 2
Temporary work dir ...........................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpeBqb0h
HMM scan output ..............................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpeBqb0h/hmm.output
HMM scan hits ................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpeBqb0h/hmm.hits
Log file .....................................: /var/folders/_1/ntm833cs6rz0br7nhdsb7rfh0000gn/T/tmpeBqb0h/00_log.txt
Number of raw hits ...........................: 4                                                                                           

#
#
# Contigs DB is ready; here are the tables in it:
#
#

collections_colors       genes_in_contigs         kmer_contigs           
collections_info         genes_in_splits          kmer_splits            
collections_of_contigs   genes_in_splits_summary  self                   
collections_of_splits    hmm_hits_in_contigs      splits_basic_info      
contig_sequences         hmm_hits_in_splits     
contigs_basic_info       hmm_hits_info          

#
#
# Profiling sample 204-6M ...
#
#

contigs_db .........................: 48 genes processed successfully.                                                                      
anvio ..............................: 1.2.2
profiler_version ...................: 6
sample_id ..........................: s204_6M
profile_db .........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-6M/PROFILE.db
contigs_db .........................: True
contigs_db_hash ....................: a70f4c26
cmd_line ...........................: /Applications/Anvio.app/Contents/Resources/base/bin/anvi-profile -i test-output/204-6M.bam -o test-output/204-6M -c test-output/CONTIGS.db
merged .............................: False
split_length .......................: 1,000
min_contig_length ..................: 5,000
min_mean_coverage ..................: 0
clustering_performed ...............: False
min_coverage_for_variability .......: 10
report_variability_full ............: False
input_bam ..........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-6M.bam                  
output_dir .........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-6M
total_reads_mapped .................: 23,783
num_contigs ........................: 3
num_contigs_after_M ................: 3
num_splits .........................: 57
total_length .......................: 56,709
profile_dict .......................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-6M/PROFILE.cp           
split_coverage_values ..............: stored in /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-6M/AUXILIARY-DATA.h5
runinfo ............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-6M/RUNINFO.cp           

#
#
# Profiling sample 204-7M ...
#
#

contigs_db .........................: 48 genes processed successfully.                                                                      
anvio ..............................: 1.2.2
profiler_version ...................: 6
sample_id ..........................: s204_7M
profile_db .........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-7M/PROFILE.db
contigs_db .........................: True
contigs_db_hash ....................: a70f4c26
cmd_line ...........................: /Applications/Anvio.app/Contents/Resources/base/bin/anvi-profile -i test-output/204-7M.bam -o test-output/204-7M -c test-output/CONTIGS.db
merged .............................: False
split_length .......................: 1,000
min_contig_length ..................: 5,000
min_mean_coverage ..................: 0
clustering_performed ...............: False
min_coverage_for_variability .......: 10
report_variability_full ............: False
input_bam ..........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-7M.bam                  
output_dir .........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-7M
total_reads_mapped .................: 5,082
num_contigs ........................: 3
num_contigs_after_M ................: 3
num_splits .........................: 57
total_length .......................: 56,709
profile_dict .......................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-7M/PROFILE.cp           
split_coverage_values ..............: stored in /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-7M/AUXILIARY-DATA.h5
runinfo ............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-7M/RUNINFO.cp           

#
#
# Profiling sample 204-9M ...
#
#

contigs_db .........................: 48 genes processed successfully.                                                                      
anvio ..............................: 1.2.2
profiler_version ...................: 6
sample_id ..........................: s204_9M
profile_db .........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-9M/PROFILE.db
contigs_db .........................: True
contigs_db_hash ....................: a70f4c26
cmd_line ...........................: /Applications/Anvio.app/Contents/Resources/base/bin/anvi-profile -i test-output/204-9M.bam -o test-output/204-9M -c test-output/CONTIGS.db
merged .............................: False
split_length .......................: 1,000
min_contig_length ..................: 5,000
min_mean_coverage ..................: 0
clustering_performed ...............: False
min_coverage_for_variability .......: 10
report_variability_full ............: False
input_bam ..........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-9M.bam                  
output_dir .........................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-9M
total_reads_mapped .................: 24,870
num_contigs ........................: 3
num_contigs_after_M ................: 3
num_splits .........................: 57
total_length .......................: 56,709
profile_dict .......................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-9M/PROFILE.cp           
split_coverage_values ..............: stored in /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-9M/AUXILIARY-DATA.h5
runinfo ............................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-9M/RUNINFO.cp           

#
#
# Merging profiles ...
#
#

total_length .................................: 56,709
num_contigs ..................................: 3
num_splits ...................................: 57
split_length .................................: 1,000
min_contig_length ............................: 5,000
min_mean_coverage ............................: 0
min_coverage_for_variability .................: 10
profiler_version .............................: 6
output_dir ...................................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-MERGED
sample_id ....................................: s204_MERGED
profile_db ...................................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-MERGED/PROFILE.db
merged .......................................: True
contigs_db_hash ..............................: a70f4c26
merged_sample_ids ............................: ['s204_6M', 's204_7M', 's204_9M']
cmd_line .....................................: /Applications/Anvio.app/Contents/Resources/base/bin/anvi-merge test-output/204-6M/RUNINFO.cp test-output/204-7M/RUNINFO.cp test-output/204-9M/RUNINFO.cp -o test-output/204-MERGED -c test-output/CONTIGS.db
num_runs_processed ...........................: 3
clustering_performed .........................: True

WARNING
===============================================
anvio just set the normalization values for each sample based on how many mapped
reads they contained. All normalized coverages will use this information:
s204_7M: 1.00, s204_6M: 0.21, s204_9M: 0.20

New hierarchical clusetring ..................: "tnf-cov" has been added to the database...                                                 
New hierarchical clusetring ..................: "tnf" has been added to the database...                                                     
New hierarchical clusetring ..................: "cov" has been added to the database...                                                     
runinfo ......................................: /Applications/Anvio.app/Contents/Resources/base/tests/sandbox/test-output/204-MERGED/RUNINFO.mcp
CONCOCT INIT .................................: Complete for 57 splits                                                                      
[09 May 16 16:42:01 VBGMM] Clustering ...                                                                                                  ./run_mini_test.sh: line 50:  1069 Illegal instruction: 4  anvi-merge test-output/204*/RUNINFO.cp -o test-output/204-MERGED -c test-output/CONTIGS.db
/run_mini_test.sh: line 50:  1069 Illegal instruction: 4  anvi-merge test-output/204*/RUNINFO.cp -o test-output/204-MERGED -c test-output/CONTIGS.db
meren commented 8 years ago

Hi Jon,

Unfortunately this makes sense :/

MAC installer contains a lot of pre-compiled programs. Although the operating system is identical, because the compiler was run on a newer version of MAC, some of the CPU specific adjustments at the compiler-level likely resulted in binaries that are incompatible with the previous version of the hardware. At least it seems to be the case with the VBGMM module, which uses scientific libraries that are heavily exploiting the CPU architecture for speed and efficiency.

If you were to recompile VBGMM on that computer and replace the .so file with the new on in the installed application directory you could hack your way out of this.

But for this case I may suggest you to follow the more conventional path to install anvi'o from source.

On the other hand this module is only used by CONCOCT. Which means the rest of the codebase will probably work without any issue, and the only thing you need to do is to use --skip-concoct-binning every time you use `anvi-merge.

Sorry about this inconvenience.

Best wishes,

jbadomics commented 8 years ago

Not a problem! Just wanted to bring it to your attention. I'll try building from source and if that proves problematic I'll just bypass CONCOCT as you recommended. Cheers! And thanks for the fast reply!