Test reference results not portable across architectures

satta commented 3 years ago

Debian runs the testsuite (make test) during the build process for its Augustus packages. While building packages for non-amd64 architectures (see list at https://buildd.debian.org/status/package.php?p=augustus), we noticed that builds were failing because the tests are run by default with --compare and results are apparently different on different architectures. So diffs fail, failing the tests and hence the builds.

I have worked around this for now by dropping --compare during the build, which fixes the immediate issue. However, I figured you might want to know about this potential issue anyway, in case you were expecting the results to be identical. Here are some example build logs, showing the diffs at the end:

MarioStanke commented 3 years ago

It seems to be for different reasons. @dhonsel , can you try to eliminate some? The case below appears to reoccur several times. Maybe this this has to do with white space differences, that could be ignored.

*** 4209 ****
! UTR exon level |          0 |              0 |       -nan |       -nan |
--- 4209 ----
! UTR exon level |          0 |              0 |        nan |        nan |
***************
*** 4211 ****
! UTR base level |          0 |              0 |       -nan |       -nan |
--- 4211 ----
! UTR base level |          0 |              0 |        nan |        nan |

satta commented 3 years ago

The case below appears to reoccur several times. Maybe this this has to do with white space differences, that could be ignored.

My bet would be on different handling of float corner cases, as we have negative NaN in one case and positive NaN in the other.

MarioStanke commented 3 years ago

Of course...

Gaius-Augustus / Augustus

Test reference results not portable across architectures #247