Closed nileshpatra closed 2 years ago
Erm the bdsim test is failing because of rounding errors at the 15th and 16th decimal places.
This no doubt has to do with the change made to edgelength precision.
Erm the bdsim test is failing because of rounding errors at the 15th and 16th decimal places.
@josephwb
Yeah, that gets fixed on building with -ffloat-store
as shared here as well
@nileshpatra did you mean -ffloat-store
?
@josephwb yeah, apologies for the typo :P
Hey @nileshpatra. I'm not sure how to go about addressing this; I'm not even sure about how to test this (beyond emulation). Do you really want to run phyx on i386? If so, I am happy to dig deeper. I only ask because this is the first request we've had.
On Sat, 24 Apr, 2021, 8:43 am Joseph W. Brown, @.***> wrote:
Hey @nileshpatra https://github.com/nileshpatra. I'm not sure how to go about addressing this; I'm not even sure about how to test this (beyond emulation).
I will be happy to test this for you
Do you really want to run phyx on i386?
Yes. I'm not a user of this package, but we maintain phyx as a Debian package. Debian officially supports i386 - and if it fails on i386, it won't get to the next release.
If so, I am happy to dig deeper. I only ask because this is the first request we've had.
You'll probably have more such requests from me in the future ;-)
Thanks for the clarification.
So, with something like -ffloat-store
, will you handle that on your end? I imagine I could set flags during configuration, but if you have in mind to deal with it I won't bother.
Thanks for the clarification.
So, with something like
-ffloat-store
, will you handle that on your end? I imagine I could set flags during configuration, but if you have in mind to deal with it I won't bother.
I will handle it, no need to modify anything to add it
Hey @nileshpatra I am finding difficult to diagnose things since I cannot even recreate the issue.
My hunch is that it has to do with nlopt
dependency, since pxstrec
(alone) uses it heavily. I am not smart enough to know whether there are flags to provide during nlopt compilation to i386-ify things?
On Wed, 28 Apr, 2021, 12:54 am Joseph W. Brown, @.***> wrote:
Hey @nileshpatra https://github.com/nileshpatra I am finding difficult to diagnose things since I cannot even recreate the issue.
Can you try to create a chroot, or maybe use qemu to recreate it?
My hunch is that it has to do with nlopt dependency, since pxstrec
(alone) uses it heavily. I am not smart enough to know whether there are flags to provide during nlopt compilation to i386-ify things?
Admittedly, I'm not aware of any such option myself :-(
Ok I will try qemu (with Ubuntu 16) but no promises ;)
Hi @josephwb! would you have any updates regarding this?
Sorry, I have a million things I need to get done first. This tab never closes, tho :)
@josephwb No worries! Just for future reference or whenever you get time, I'm adding this information -- in case if it might help:
I added two lines to get what kind of file is in the output of pxstrec test
cm = "./pxstrec -t TEST/test.strec.tre -d TEST/test.data.narrow -c TEST/test.conf | tail -1"
+ print("=================== OUTPUT FAULTY =====================")
+ os.system("./pxstrec -t TEST/test.strec.tre -d TEST/test.data.narrow -c TEST/test.conf")
t = '0.316161 \n'
And in i386 test, I get many nan values, so there's some kind of underflow happening. Here's the output that I get:
TESTING pxstrec
=================== OUTPUT FAULTY =====================
finished reading config file
taxa: 10
trees: 1
nsites: 2
total number of states in dataset: 4
site: 1
states: 4
trees: 0
tips: 10
initializing nodes...
taxon_4 1000
taxon_5 1000
taxon_6 1000
taxon_3 0100
taxon_7 0010
taxon_8 0010
taxon_9 0010
taxon_10 0010
taxon_11 0001
taxon_12 0001
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
1
likelihood: 11.9006
0 0.0441 0.0441 0.0441
0.0441 0 0.0441 0.0441
0.0441 0.0441 0 0.0441
0.0441 0.0441 0.0441 0
final_likelihood: 8.92926
ancestral states
stochastic time
mrca: 9_4
nan nan nan nan
stochastic number
mrca: 9_4
- nan nan nan
nan - nan nan
nan nan - nan
nan nan nan -
stochastic number (any)
node: 2 mrca: 9_4
nan
site: 2
states: 4
trees: 0
tips: 10
initializing nodes...
taxon_4 1000
taxon_5 1000
taxon_6 1000
taxon_3 0100
taxon_7 0100
taxon_8 0010
taxon_9 0010
taxon_10 0010
taxon_11 0001
taxon_12 0001
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
1
likelihood: 11.9952
0 0.0727 0.0727 0.0727
0.0727 0 0.0727 0.0727
0.0727 0.0727 0 0.0727
0.0727 0.0727 0.0727 0
final_likelihood: 10.751
ancestral states
stochastic time
mrca: 9_4
nan nan nan nan
stochastic number
mrca: 9_4
- nan nan nan
nan - nan nan
nan nan - nan
nan nan nan -
stochastic number (any)
node: 2 mrca: 9_4
nan
n 0 False
a . False
n 3 False
1 False
6 False
FAILED
intended output:
0.316161
your output:
nan
Ok, that seems helpful. Thanks.
Sorry I have not yet gotten to this. But there was a major overhaul recently. It is probably useful to pull/rebuild to see if additional tests now fail \(〇_o)/
Did so, this is what I get:
=================
TESTING pxaa2cdn
PASSED
=================
TESTING pxbdfit
yule PASSED
birth-death PASSED
best PASSED
PASSED
=================
TESTING pxbdsim
taxon stop ( ( True
( ( True
t t True
a a True
x x True
o o True
n n True
_ _ True
1 1 True
: : True
0 0 True
. . True
3 3 True
0 0 True
8 8 True
5 5 True
5 5 True
6 6 True
9 9 True
3 3 True
4 4 True
2 2 True
6 6 True
1 1 True
0 0 True
9 9 True
6 6 True
4 5 False
, , True
t t True
a a True
x x True
o o True
n n True
_ _ True
2 2 True
: : True
0 0 True
. . True
3 3 True
0 0 True
8 8 True
5 5 True
5 5 True
6 6 True
9 9 True
3 3 True
4 4 True
2 2 True
6 6 True
1 1 True
0 0 True
9 9 True
6 6 True
4 5 False
) ) True
: : True
0 0 True
. . True
6 6 True
3 3 True
1 1 True
3 3 True
7 7 True
1 1 True
0 0 True
4 4 True
6 6 True
2 2 True
7 7 True
1 1 True
9 9 True
9 9 True
0 0 True
6 6 True
, , True
( ( True
t t True
a a True
x x True
o o True
n n True
_ _ True
3 3 True
: : True
0 0 True
. . True
9 9 True
3 3 True
8 8 True
5 5 True
1 1 True
8 8 True
3 3 True
9 9 True
9 9 True
5 5 True
9 9 True
5 5 True
6 6 True
3 3 True
3 3 True
5 7 False
, , True
( ( True
t t True
a a True
x x True
o o True
n n True
_ _ True
4 4 True
: : True
0 0 True
. . True
5 5 True
3 3 True
9 9 True
7 7 True
7 7 True
5 5 True
8 8 True
4 4 True
9 9 True
6 6 True
9 9 True
8 8 True
8 8 True
5 5 True
8 9 False
9 1 False
, , True
t t True
a a True
x x True
o o True
n n True
_ _ True
5 5 True
: : True
0 0 True
. . True
5 5 True
3 3 True
9 9 True
7 7 True
7 7 True
5 5 True
8 8 True
4 4 True
9 9 True
6 6 True
9 9 True
8 8 True
8 8 True
5 5 True
8 9 False
9 1 False
) ) True
: : True
0 0 True
. . True
3 3 True
9 9 True
8 8 True
7 7 True
4 4 True
2 2 True
5 5 True
4 4 True
9 9 True
8 8 True
9 9 True
6 6 True
7 7 True
7 7 True
4 4 True
7 7 True
) ) True
: : True
0 0 True
. . True
0 0 True
0 0 True
1 1 True
4 4 True
0 0 True
9 9 True
5 5 True
8 8 True
0 0 True
9 9 True
3 3 True
7 7 True
4 4 True
5 5 True
3 3 True
5 4 False
) ) True
; ; True
True
FAILED
intended output:
((taxon_1:0.3085569342610965,taxon_2:0.3085569342610965):0.6313710462719906,(taxon_3:0.9385183995956337,(taxon_4:0.5397758496988591,taxon_5:0.5397758496988591):0.3987425498967747):0.0014095809374534);
your output:
((taxon_1:0.3085569342610964,taxon_2:0.3085569342610964):0.6313710462719906,(taxon_3:0.9385183995956335,(taxon_4:0.5397758496988589,taxon_5:0.5397758496988589):0.3987425498967747):0.0014095809374535);
time stop PASSED
FAILED
=================
TESTING pxboot
bootstrap PASSED
jackknife PASSED
PASSED
=================
TESTING pxbp
PASSED
=================
TESTING pxcat
PASSED
=================
TESTING pxclsq
delete sites PASSED
info by taxon PASSED
PASSED
=================
TESTING pxcltr
all PASSED
labels PASSED
knuckles PASSED
PASSED
=================
TESTING pxcolt
probability support PASSED
integer support PASSED
real support PASSED
PASSED
=================
TESTING pxcomp
PASSED
=================
TESTING pxconsq
PASSED
=================
TESTING pxcontrates
character: 0
character: 1
character: 2
PASSED
=================
TESTING pxfqfilt
PASSED
=================
TESTING pxlog
count trees PASSED
tree log PASSED
count parameters PASSED
parameter log PASSED
PASSED
=================
TESTING pxlssq
default PASSED
labels PASSED
frequencies PASSED
indidivual stats PASSED
binary data PASSED
morph data PASSED
fastq PASSED
unaliagned PASSED
individual missing PASSED
PASSED
=================
TESTING pxlstr
newick PASSED
nexus (mulitple) PASSED
figtree PASSED
polytomy PASSED
internal linebreak PASSED
PASSED
=================
TESTING pxmono
PASSED
=================
TESTING pxmrca
PASSED
=================
TESTING pxmrcacut
PASSED
=================
TESTING pxmrcaname
PASSED
=================
TESTING pxnw
PASSED
=================
TESTING pxpoly
PASSED
=================
TESTING pxrecode
RY-coding PASSED
MK-coding PASSED
SW-coding PASSED
PASSED
=================
TESTING pxrevcomp
PASSED
=================
TESTING pxrls
PASSED
=================
TESTING pxrlt
PASSED
=================
TESTING pxrmk
PASSED
=================
TESTING pxrms
command line arg PASSED
regex PASSED
PASSED
=================
TESTING pxrmt
command line arg PASSED
regex PASSED
PASSED
=================
TESTING pxrr
PASSED
=================
TESTING pxs2fa
nexus PASSED
interleaved nexus PASSED
phylip PASSED
interleaved phylip PASSED
fastq PASSED
PASSED
=================
TESTING pxs2nex
fasta PASSED
phylip PASSED
interleaved phylip PASSED
interleaved nexus PASSED
fastq PASSED
PASSED
=================
TESTING pxs2phy
fasta PASSED
nexus PASSED
interleaved nexus PASSED
interleaved phylip PASSED
PASSED
=================
TESTING pxseqgen
PASSED
=================
TESTING pxssort
sort by id PASSED
sort by id rev PASSED
sort by length (<) PASSED
sort by length (>) PASSED
PASSED
=================
TESTING pxsstat
PASSED
=================
TESTING pxstrec
n 0 False
a . False
n 3 False
1 False
6 False
FAILED
intended output:
0.316161
your output:
nan
=================
TESTING pxsw
PASSED
=================
TESTING pxt2new
PASSED
=================
TESTING pxt2nex
PASSED
=================
TESTING pxtcol
no test for pxtcol
=================
TESTING pxtcomb
no test for pxtcomb
=================
TESTING pxtgen
unrooted PASSED
rooted PASSED
prefix label PASSED
PASSED
=================
TESTING pxtlate
standard PASSED
vert mtDNA PASSED
PASSED
=================
TESTING pxtrt
PASSED
=================
TESTING pxtscale
scale PASSED
root height PASSED
PASSED
=================
TESTING pxvcf2fa
PASSED
=================
PASSED PROGRAMS: 43 (89 tests)
FAILED PROGRAMS: 1 (2 tests)
PROGRAMS WITHOUT TESTS: 2
The following programs had anticipated failures (random numbers):
pxbdsim
These failed:
pxstrec
Thanks for checking. I'm glad nothing new was broken!
Hi, I wonder whether it makes finally sense to declare phyx as 64 bit only. This probably reflects its real use case. In Debian packages are build on older architectures and while this has no practical use cases in many cases like bioinformatics it sometimes helps to spot issues inside the code. If we can be sure that this is not the case here it might help to state clearly that 32 bit architectures are not supported and than we can exclude these and be done with the issue. Kind regards, Andreas.
Could you please confirm whether restriction to 64 bit only makes sense or not?
Hi, about the remaining nan error in pxstrec, I believe I tracked it down to _StateReconstructor::calculate_reversestochmap and to the use of -ffast-math build option. When I build _src/statereconstructor.cpp with -ffloat-store in place of -ffast-math, then the remaining test item pass successfully on my end on i686 machines.
I'm working around all build and test issues with the below patch:
--- phyx.orig/src/Makefile.in
+++ phyx/src/Makefile.in
@@ -32,7 +32,12 @@
endif
ifneq "$(CXX)" "icc"
- OPT_FLAGS += -ffast-math -ftree-vectorize
+ OPT_FLAGS += -ftree-vectorize
+ ifneq ($(filter @host_cpu@,i386 i486 i586 i686),)
+ OPT_FLAGS += -ffloat-store
+ else
+ OPT_FLAGS += -ffast-math
+ endif
endif
NLOPT_PROGRAMS :=
In hope this helps, Have a nice day, :) Étienne.
Since it builds for us now, I am closing the issue.
This was passing in version 1.01, please consider fixing this regression