josephwb
commented
3 years ago Erm the bdsim test is failing because of rounding errors at the 15th and 16th decimal places.
This no doubt has to do with the change made to edgelength precision.
Closed nileshpatra closed 2 years ago
josephwb
commented
3 years ago Erm the bdsim test is failing because of rounding errors at the 15th and 16th decimal places.
This no doubt has to do with the change made to edgelength precision.
nileshpatra
commented
3 years ago Erm the bdsim test is failing because of rounding errors at the 15th and 16th decimal places.
@josephwb
Yeah, that gets fixed on building with -ffloat-store as shared here as well
josephwb
commented
3 years ago @nileshpatra did you mean -ffloat-store?
nileshpatra
commented
3 years ago @josephwb yeah, apologies for the typo :P
josephwb
commented
3 years ago Hey @nileshpatra. I'm not sure how to go about addressing this; I'm not even sure about how to test this (beyond emulation). Do you really want to run phyx on i386? If so, I am happy to dig deeper. I only ask because this is the first request we've had.
nileshpatra
commented
3 years ago On Sat, 24 Apr, 2021, 8:43 am Joseph W. Brown, @.***> wrote:
Hey @nileshpatra https://github.com/nileshpatra. I'm not sure how to go about addressing this; I'm not even sure about how to test this (beyond emulation).
I will be happy to test this for you
Do you really want to run phyx on i386?
Yes. I'm not a user of this package, but we maintain phyx as a Debian package. Debian officially supports i386 - and if it fails on i386, it won't get to the next release.
If so, I am happy to dig deeper. I only ask because this is the first request we've had.
You'll probably have more such requests from me in the future ;-)
josephwb
commented
3 years ago Thanks for the clarification.
So, with something like -ffloat-store, will you handle that on your end? I imagine I could set flags during configuration, but if you have in mind to deal with it I won't bother.
nileshpatra
commented
3 years ago Thanks for the clarification.
So, with something like
-ffloat-store, will you handle that on your end? I imagine I could set flags during configuration, but if you have in mind to deal with it I won't bother.
I will handle it, no need to modify anything to add it
josephwb
commented
3 years ago Hey @nileshpatra I am finding difficult to diagnose things since I cannot even recreate the issue.
My hunch is that it has to do with nlopt dependency, since pxstrec (alone) uses it heavily. I am not smart enough to know whether there are flags to provide during nlopt compilation to i386-ify things?
nileshpatra
commented
3 years ago On Wed, 28 Apr, 2021, 12:54 am Joseph W. Brown, @.***> wrote:
Hey @nileshpatra https://github.com/nileshpatra I am finding difficult to diagnose things since I cannot even recreate the issue.
Can you try to create a chroot, or maybe use qemu to recreate it?
My hunch is that it has to do with nlopt dependency, since pxstrec
(alone) uses it heavily. I am not smart enough to know whether there are flags to provide during nlopt compilation to i386-ify things?
Admittedly, I'm not aware of any such option myself :-(
josephwb
commented
3 years ago Ok I will try qemu (with Ubuntu 16) but no promises ;)
nileshpatra
commented
3 years ago Hi @josephwb! would you have any updates regarding this?
josephwb
commented
3 years ago Sorry, I have a million things I need to get done first. This tab never closes, tho :)
nileshpatra
commented
3 years ago @josephwb No worries! Just for future reference or whenever you get time, I'm adding this information -- in case if it might help:
I added two lines to get what kind of file is in the output of pxstrec test
cm = "./pxstrec -t TEST/test.strec.tre -d TEST/test.data.narrow -c TEST/test.conf | tail -1"
+ print("=================== OUTPUT FAULTY =====================")
+ os.system("./pxstrec -t TEST/test.strec.tre -d TEST/test.data.narrow -c TEST/test.conf")
t = '0.316161 \n'And in i386 test, I get many nan values, so there's some kind of underflow happening. Here's the output that I get:
TESTING pxstrec
=================== OUTPUT FAULTY =====================
finished reading config file
taxa: 10
trees: 1
nsites: 2
total number of states in dataset: 4
site: 1
states: 4
trees: 0
tips: 10
initializing nodes...
taxon_4 1000
taxon_5 1000
taxon_6 1000
taxon_3 0100
taxon_7 0010
taxon_8 0010
taxon_9 0010
taxon_10 0010
taxon_11 0001
taxon_12 0001
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
1
likelihood: 11.9006
0 0.0441 0.0441 0.0441
0.0441 0 0.0441 0.0441
0.0441 0.0441 0 0.0441
0.0441 0.0441 0.0441 0
final_likelihood: 8.92926
ancestral states
stochastic time
mrca: 9_4
nan nan nan nan
stochastic number
mrca: 9_4
- nan nan nan
nan - nan nan
nan nan - nan
nan nan nan -
stochastic number (any)
node: 2 mrca: 9_4
nan
site: 2
states: 4
trees: 0
tips: 10
initializing nodes...
taxon_4 1000
taxon_5 1000
taxon_6 1000
taxon_3 0100
taxon_7 0100
taxon_8 0010
taxon_9 0010
taxon_10 0010
taxon_11 0001
taxon_12 0001
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
1
likelihood: 11.9952
0 0.0727 0.0727 0.0727
0.0727 0 0.0727 0.0727
0.0727 0.0727 0 0.0727
0.0727 0.0727 0.0727 0
final_likelihood: 10.751
ancestral states
stochastic time
mrca: 9_4
nan nan nan nan
stochastic number
mrca: 9_4
- nan nan nan
nan - nan nan
nan nan - nan
nan nan nan -
stochastic number (any)
node: 2 mrca: 9_4
nan
n 0 False
a . False
n 3 False
1 False
6 False
FAILED
intended output:
0.316161
your output:
nan Ok, that seems helpful. Thanks.
josephwb
commented
3 years ago Sorry I have not yet gotten to this. But there was a major overhaul recently. It is probably useful to pull/rebuild to see if additional tests now fail ＼(〇_ｏ)／
nileshpatra
commented
3 years ago Did so, this is what I get:
=================
TESTING pxaa2cdn
PASSED
=================
TESTING pxbdfit
yule PASSED
birth-death PASSED
best PASSED
PASSED
=================
TESTING pxbdsim
taxon stop ( ( True
( ( True
t t True
a a True
x x True
o o True
n n True
_ _ True
1 1 True
: : True
0 0 True
. . True
3 3 True
0 0 True
8 8 True
5 5 True
5 5 True
6 6 True
9 9 True
3 3 True
4 4 True
2 2 True
6 6 True
1 1 True
0 0 True
9 9 True
6 6 True
4 5 False
, , True
t t True
a a True
x x True
o o True
n n True
_ _ True
2 2 True
: : True
0 0 True
. . True
3 3 True
0 0 True
8 8 True
5 5 True
5 5 True
6 6 True
9 9 True
3 3 True
4 4 True
2 2 True
6 6 True
1 1 True
0 0 True
9 9 True
6 6 True
4 5 False
) ) True
: : True
0 0 True
. . True
6 6 True
3 3 True
1 1 True
3 3 True
7 7 True
1 1 True
0 0 True
4 4 True
6 6 True
2 2 True
7 7 True
1 1 True
9 9 True
9 9 True
0 0 True
6 6 True
, , True
( ( True
t t True
a a True
x x True
o o True
n n True
_ _ True
3 3 True
: : True
0 0 True
. . True
9 9 True
3 3 True
8 8 True
5 5 True
1 1 True
8 8 True
3 3 True
9 9 True
9 9 True
5 5 True
9 9 True
5 5 True
6 6 True
3 3 True
3 3 True
5 7 False
, , True
( ( True
t t True
a a True
x x True
o o True
n n True
_ _ True
4 4 True
: : True
0 0 True
. . True
5 5 True
3 3 True
9 9 True
7 7 True
7 7 True
5 5 True
8 8 True
4 4 True
9 9 True
6 6 True
9 9 True
8 8 True
8 8 True
5 5 True
8 9 False
9 1 False
, , True
t t True
a a True
x x True
o o True
n n True
_ _ True
5 5 True
: : True
0 0 True
. . True
5 5 True
3 3 True
9 9 True
7 7 True
7 7 True
5 5 True
8 8 True
4 4 True
9 9 True
6 6 True
9 9 True
8 8 True
8 8 True
5 5 True
8 9 False
9 1 False
) ) True
: : True
0 0 True
. . True
3 3 True
9 9 True
8 8 True
7 7 True
4 4 True
2 2 True
5 5 True
4 4 True
9 9 True
8 8 True
9 9 True
6 6 True
7 7 True
7 7 True
4 4 True
7 7 True
) ) True
: : True
0 0 True
. . True
0 0 True
0 0 True
1 1 True
4 4 True
0 0 True
9 9 True
5 5 True
8 8 True
0 0 True
9 9 True
3 3 True
7 7 True
4 4 True
5 5 True
3 3 True
5 4 False
) ) True
; ; True
True
FAILED
intended output:
((taxon_1:0.3085569342610965,taxon_2:0.3085569342610965):0.6313710462719906,(taxon_3:0.9385183995956337,(taxon_4:0.5397758496988591,taxon_5:0.5397758496988591):0.3987425498967747):0.0014095809374534);
your output:
((taxon_1:0.3085569342610964,taxon_2:0.3085569342610964):0.6313710462719906,(taxon_3:0.9385183995956335,(taxon_4:0.5397758496988589,taxon_5:0.5397758496988589):0.3987425498967747):0.0014095809374535);
time stop PASSED
FAILED
=================
TESTING pxboot
bootstrap PASSED
jackknife PASSED
PASSED
=================
TESTING pxbp
PASSED
=================
TESTING pxcat
PASSED
=================
TESTING pxclsq
delete sites PASSED
info by taxon PASSED
PASSED
=================
TESTING pxcltr
all PASSED
labels PASSED
knuckles PASSED
PASSED
=================
TESTING pxcolt
probability support PASSED
integer support PASSED
real support PASSED
PASSED
=================
TESTING pxcomp
PASSED
=================
TESTING pxconsq
PASSED
=================
TESTING pxcontrates
character: 0
character: 1
character: 2
PASSED
=================
TESTING pxfqfilt
PASSED
=================
TESTING pxlog
count trees PASSED
tree log PASSED
count parameters PASSED
parameter log PASSED
PASSED
=================
TESTING pxlssq
default PASSED
labels PASSED
frequencies PASSED
indidivual stats PASSED
binary data PASSED
morph data PASSED
fastq PASSED
unaliagned PASSED
individual missing PASSED
PASSED
=================
TESTING pxlstr
newick PASSED
nexus (mulitple) PASSED
figtree PASSED
polytomy PASSED
internal linebreak PASSED
PASSED
=================
TESTING pxmono
PASSED
=================
TESTING pxmrca
PASSED
=================
TESTING pxmrcacut
PASSED
=================
TESTING pxmrcaname
PASSED
=================
TESTING pxnw
PASSED
=================
TESTING pxpoly
PASSED
=================
TESTING pxrecode
RY-coding PASSED
MK-coding PASSED
SW-coding PASSED
PASSED
=================
TESTING pxrevcomp
PASSED
=================
TESTING pxrls
PASSED
=================
TESTING pxrlt
PASSED
=================
TESTING pxrmk
PASSED
=================
TESTING pxrms
command line arg PASSED
regex PASSED
PASSED
=================
TESTING pxrmt
command line arg PASSED
regex PASSED
PASSED
=================
TESTING pxrr
PASSED
=================
TESTING pxs2fa
nexus PASSED
interleaved nexus PASSED
phylip PASSED
interleaved phylip PASSED
fastq PASSED
PASSED
=================
TESTING pxs2nex
fasta PASSED
phylip PASSED
interleaved phylip PASSED
interleaved nexus PASSED
fastq PASSED
PASSED
=================
TESTING pxs2phy
fasta PASSED
nexus PASSED
interleaved nexus PASSED
interleaved phylip PASSED
PASSED
=================
TESTING pxseqgen
PASSED
=================
TESTING pxssort
sort by id PASSED
sort by id rev PASSED
sort by length (<) PASSED
sort by length (>) PASSED
PASSED
=================
TESTING pxsstat
PASSED
=================
TESTING pxstrec
n 0 False
a . False
n 3 False
1 False
6 False
FAILED
intended output:
0.316161
your output:
nan
=================
TESTING pxsw
PASSED
=================
TESTING pxt2new
PASSED
=================
TESTING pxt2nex
PASSED
=================
TESTING pxtcol
no test for pxtcol
=================
TESTING pxtcomb
no test for pxtcomb
=================
TESTING pxtgen
unrooted PASSED
rooted PASSED
prefix label PASSED
PASSED
=================
TESTING pxtlate
standard PASSED
vert mtDNA PASSED
PASSED
=================
TESTING pxtrt
PASSED
=================
TESTING pxtscale
scale PASSED
root height PASSED
PASSED
=================
TESTING pxvcf2fa
PASSED
=================
PASSED PROGRAMS: 43 (89 tests)
FAILED PROGRAMS: 1 (2 tests)
PROGRAMS WITHOUT TESTS: 2
The following programs had anticipated failures (random numbers):
pxbdsim
These failed:
pxstrecThanks for checking. I'm glad nothing new was broken!
tillea
commented
2 years ago Hi, I wonder whether it makes finally sense to declare phyx as 64 bit only. This probably reflects its real use case. In Debian packages are build on older architectures and while this has no practical use cases in many cases like bioinformatics it sometimes helps to spot issues inside the code. If we can be sure that this is not the case here it might help to state clearly that 32 bit architectures are not supported and than we can exclude these and be done with the issue. Kind regards, Andreas.
tillea
commented
2 years ago Could you please confirm whether restriction to 64 bit only makes sense or not?
emollier
commented
2 years ago Hi, about the remaining nan error in pxstrec, I believe I tracked it down to _StateReconstructor::calculate_reversestochmap and to the use of -ffast-math build option. When I build _src/statereconstructor.cpp with -ffloat-store in place of -ffast-math, then the remaining test item pass successfully on my end on i686 machines.
I'm working around all build and test issues with the below patch:
--- phyx.orig/src/Makefile.in
+++ phyx/src/Makefile.in
@@ -32,7 +32,12 @@
endif
ifneq "$(CXX)" "icc"
- OPT_FLAGS += -ffast-math -ftree-vectorize
+ OPT_FLAGS += -ftree-vectorize
+ ifneq ($(filter @host_cpu@,i386 i486 i586 i686),)
+ OPT_FLAGS += -ffloat-store
+ else
+ OPT_FLAGS += -ffast-math
+ endif
endif
NLOPT_PROGRAMS :=In hope this helps, Have a nice day, :) Étienne.
nileshpatra
commented
2 years ago Since it builds for us now, I am closing the issue.
This was passing in version 1.01, please consider fixing this regression