Conda installation not working

[MaestSi]

Hi, spoa conda installation exited with a segmentation fault (core dump) error when running it with a very small dataset composed of 600bp-long reads. After installing spoa from Github, it worked fine. May the issue be related to this from racon? Thanks, Simone

[rvaser]

Hello Simone, did you get illegal instruction error or just the segmentation fault?

Best regards, Robert

[rvaser]

Ah, I saw that you were able to run it when you compile it from source. Then yes, it is probably tied to -march=native flag used for the Conda build. I did not have yet the time to deal with it.

[ksahlin]

I can second this as our software that depends on spoa (v3.0.1) breaks when installed from conda (conda install --yes -c conda-forge -c bioconda spoa). Is there a workaround/fix for this?

I get:

(NGSpeciesID) [ksahlin@brubeck source]$ spoa /tmp/tmpws2qldao/reads_c_id_17.fq -l 0 -r 0 -g -2
Illegal instruction```

[rvaser]

I did not change anything recently. Did you successfully use v3.0.1 from conda before?

Best regards, Robert

[ksahlin]

Yes, Here is the one working (not installed through conda):

[ksahlin@brubeck source]$ which spoa
/galaxy/home/ksahlin/prefix/bin/spoa
[ksahlin@brubeck source]$ spoa --version
v3.0.1
[ksahlin@brubeck source]$ spoa /tmp/tmpws2qldao/reads_c_id_17.fq -l 0 -r 0 -g -2
Consensus (670)
AAAGAATAGAATAAATGTTGATATAAAATGGGGTCACCTCCTCCAGCAGGATCAAAAAAAAAGTATTGAGGTTACGATCAGTTAAAAGTATTGTGATACCGGCAGCTAAGACAGGAAGAGATAATAGCAGAAGCACAGCTGTAGTAGGACTGCTCAAACAAAAAGGGTGTTTGATATTGTGAAATTGCAGGAGGTTTTATATTAATAATTGTGGTAATAAAATTAATTGCTCCAGAATTGATGAAATTCCGGCTAGATGAAGAAAAAATAGTGAGGTCAACAGATGCCCCAGCATGAGCGAGGTTTCCAGCTAAAGGGGATAAACAGTTCAGCCTGTGCCAGCTCAGCTTCTACTCAGATGATGCTAAAAGAAGCAGAAAAGATGGAGGAAGAAGTCAGAAACTTATGTTGTTTATTCGAGGAAAGCCATATCTGGGGCACCAATCATTAGAGGAATGAGTCAGTTTCGAAGCCTCAATTATTAGTGGTATTACTATAAAGAAAATTATTACGAAAGCGTGTGCTGTGACAATCACATTATAAATTTGATCATCACCTAGAAGTGCACCAGGTTGACTTAGCTCTGCTCGAATGAGCAGGCTTAAGGCTGTTCCGACTATGCCGGCTCAGGCACCAAATACGAGATAAAGGGTGCCGATAGTCTTGTAGT

Here is the one installed with conda today:

[ksahlin@brubeck source]$ conda activate NGSpeciesID
(NGSpeciesID) [ksahlin@brubeck source]$ which spoa
/galaxy/home/ksahlin/prefix/anaconda2/envs/NGSpeciesID/bin/spoa
(NGSpeciesID) [ksahlin@brubeck source]$ spoa --version
v3.0.1
(NGSpeciesID) [ksahlin@brubeck source]$ spoa /tmp/tmpws2qldao/reads_c_id_17.fq -l 0 -r 0 -g -2
Illegal instruction

[rvaser]

I have crated a pull request to Bioconda which will build spoa with -msse4.1 from now on. Please try it out once it is merged (I will drop a notification here).

[MaestSi]

Thanks, I'll try it out once it is ready. Would it be possible for you to do the same for racon easily? Simone

[rvaser]

If this will work as planned, I will update Racon accordingly.

The update was merged into Bioconda master branch, but it is not yet available for download.

[rvaser]

@MaestSi and @ksahlin, can you please update your spoa package in bioconda and try if it works now?

[MaestSi]

Hi, I have downloaded the conda version and it works, thank you very much. Hope you are finding the time for doing the same with racon. :) Simone

[ksahlin]

It doesn't seem to work for me yet (it now has the v3.0.2). I followed instructions here and did

conda create -n NGSpeciesID python=3.6 pip 
conda activate NGSpeciesID
pip install NGSpeciesID
conda install --yes -c conda-forge -c bioconda medaka openblas==0.3.3 spoa

Results in:

NGSpeciesID) [ksahlin@brubeck ksahlin]$ which spoa
/galaxy/home/ksahlin/prefix/anaconda2/envs/NGSpeciesID/bin/spoa
(NGSpeciesID) [ksahlin@brubeck ksahlin]$ spoa --version
v3.0.2

The error:

subprocess.CalledProcessError: Command '['spoa', '/tmp/tmpizjpkduz/reads_c_id_17.fq', '-l', '0', '-r', '0', '-g', '-2']' died with <Signals.SIGILL: 4>.

and the spoa command run directly in terminal:

(NGSpeciesID) [ksahlin@brubeck ksahlin]$ spoa /tmp/tmpizjpkduz/reads_c_id_17.fq -l 0 -r 0 -g -2
Illegal instruction

Any ideas?

[rvaser]

Which CPU do you have? Does it support sse4.1?

[ksahlin]

I tried some commands below. It says sse sse2 sse4a. Does this give the information? If it's not supported, could I overcome this on machines where sse4.1 is not supported?

(NGSpeciesID) [ksahlin@brubeck ksahlin]$ cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
(NGSpeciesID) [ksahlin@brubeck ksahlin]$ uname -r
4.19.0-6-amd64

(NGSpeciesID) [ksahlin@brubeck ksahlin]$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       48 bits physical, 48 bits virtual
CPU(s):              48
On-line CPU(s) list: 0-47
Thread(s) per core:  1
Core(s) per socket:  12
Socket(s):           4
NUMA node(s):        8
Vendor ID:           AuthenticAMD
CPU family:          16
Model:               9
Model name:          AMD Opteron(tm) Processor 6174
Stepping:            1
CPU MHz:             2200.119
BogoMIPS:            4400.23
Virtualization:      AMD-V
L1d cache:           64K
L1i cache:           64K
L2 cache:            512K
L3 cache:            5118K
NUMA node0 CPU(s):   0-5
NUMA node1 CPU(s):   6-11
NUMA node2 CPU(s):   36-41
NUMA node3 CPU(s):   42-47
NUMA node4 CPU(s):   24-29
NUMA node5 CPU(s):   30-35
NUMA node6 CPU(s):   12-17
NUMA node7 CPU(s):   18-23
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr hw_pstate vmmcall npt lbrv svm_lock nrip_save pausefilter

(NGSpeciesID) [ksahlin@brubeck ksahlin]$ cat /proc/cpuinfo 
processor   : 0
vendor_id   : AuthenticAMD
cpu family  : 16
model       : 9
model name  : AMD Opteron(tm) Processor 6174
stepping    : 1
microcode   : 0x10000d9
cpu MHz     : 2200.119
cache size  : 512 KB
physical id : 0
siblings    : 12
core id     : 0
cpu cores   : 12
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 5
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr hw_pstate vmmcall npt lbrv svm_lock nrip_save pausefilter
bugs        : tlb_mmatch fxsave_leak sysret_ss_attrs null_seg spectre_v1 spectre_v2
bogomips    : 4400.23
TLB size    : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

[rvaser]

As it seems it does not support sse4.1 (only sse4a which has a tiny fraction of the instructions). It is a bit old processor don't you think? Can you try on a newer one? When you compile from source, it will work fine as Spoa falls back to non-vectorized code if no compatible instruction set is present.

[ksahlin]

hehe maybe, it is what I've been given to work with :) I will check with a collaborator if he can install it. But I guess we have found the cause of the problem at least.

[ksahlin]

I checked with my collaborator and he installed the new version on both an Ubuntu and Centos and both worked like a charm so all good! Thanks!

[rvaser]

That is great! May I close this issue then?

[ksahlin]

fine for me!