smarco / WFA2-lib

WFA-lib: Wavefront alignment algorithm library v2
Other
162 stars 36 forks source link

Please support big-endian architectures #79

Closed vpa1977 closed 1 year ago

vpa1977 commented 1 year ago

Hi,

Would it be possible to consider supporting BIG_ENDIAN architectures such as S390X ?

Running tests for them at the moment results in failures due to the assumptions about the byte order:

Comparing ./tests vs ./tests/wfa.utest.check
[UTest::test.affine] Error
[UTest::test.affine.p0] Error
[UTest::test.affine.p1] Error
[UTest::test.affine.p2] Error
[UTest::test.affine.p3] Error
[UTest::test.affine.p4] Error
[UTest::test.affine.p5] Error
[UTest::test.affine.wfapt0] Error
[UTest::test.affine.wfapt1] Error
[UTest::test.affine2p] Error
[UTest::test.biwfa.affine] Error
[UTest::test.biwfa.affine.p0] Error
[UTest::test.biwfa.affine.p1] Error
[UTest::test.biwfa.affine.p2] Error
[UTest::test.biwfa.affine.p3] Error
[UTest::test.biwfa.affine.p4] Error
[UTest::test.biwfa.affine.p5] Error
[UTest::test.biwfa.affine.wfapt0] Error
[UTest::test.biwfa.affine.wfapt1] Error
[UTest::test.biwfa.affine2p] Error
[UTest::test.biwfa.edit] Error
[UTest::test.biwfa.indel] Error
[UTest::test.biwfa.score.affine] Error
[UTest::test.biwfa.score.affine.p0] Error
[UTest::test.biwfa.score.affine.p1] Error
[UTest::test.biwfa.score.affine.p2] Error
[UTest::test.biwfa.score.affine.p3] Error
[UTest::test.biwfa.score.affine.p4] Error
[UTest::test.biwfa.score.affine.p5] Error
[UTest::test.biwfa.score.affine.wfapt0] Error
[UTest::test.biwfa.score.affine.wfapt1] Error
[UTest::test.biwfa.score.affine2p] Error
[UTest::test.biwfa.score.edit] Error
[UTest::test.biwfa.score.indel] Error
[UTest::test.edit] Error
[UTest::test.indel] Error
[UTest::test.pb.affine] Error
[UTest::test.pb.affine.p0] Error
[UTest::test.pb.affine.p1] Error
[UTest::test.pb.affine.p2] Error
[UTest::test.pb.affine.p3] Error
[UTest::test.pb.affine.p4] Error
[UTest::test.pb.affine.p5] Error
[UTest::test.pb.affine.wfapt0] Error
[UTest::test.pb.affine.wfapt1] Error
[UTest::test.pb.affine2p] Error
[UTest::test.pb.edit] Error
[UTest::test.pb.indel] Error
[UTest::test.score.affine] Error
[UTest::test.score.affine.p0] Error
[UTest::test.score.affine.p1] Error
[UTest::test.score.affine.p2] Error
[UTest::test.score.affine.p3] Error
[UTest::test.score.affine.p4] Error
[UTest::test.score.affine.p5] Error
[UTest::test.score.affine.wfapt0] Error
[UTest::test.score.affine.wfapt1] Error
[UTest::test.score.affine2p] Error
[UTest::test.score.edit] Error
[UTest::test.score.indel] Error
smarco commented 1 year ago

Hi,

If the arch is not LITTLE-ENDIAN, I've tried to implement a patch that makes no assumption and resorts to a much simpler and less efficient implementation. I would appreciate it if you could try it on your BIG-ENDIAN machine (I would like to avoid building a qemu BIG-ENDIAN ARM). Let me know.

vpa1977 commented 1 year ago

Hi,

I've re run the test on s390x:

$ ./tests/wfa.utest.sh 
>>> Testing 'test' (--check=correct)
>>> Testing 'test.score' (--wfa-score-only)
>>> Testing 'test.pb' (--wfa-memory-mode=med --check=correct)
>>> Testing 'test.biwfa' (--wfa-memory-mode=ultralow --check=correct)
>>> Testing 'test.biwfa.score' (--wfa-memory-mode=ultralow --wfa-score-only)
>>> Performance Time (s): 
2.22 s  8.44 s
2.76 s  8.59 s
3.80 s  9.37 s
7.81 s  32.84 s
>>> Performance Mem (KB): 
410692  410240
558932  558336
827556  826880
1783444 1782656
> Comparing ./tests vs ./tests/wfa.utest.check
[UTest::test.affine]    OK
[UTest::test.affine.p0]     OK
[UTest::test.affine.p1]     OK
[UTest::test.affine.p2]     OK
[UTest::test.affine.p3]     OK
[UTest::test.affine.p4]     OK
[UTest::test.affine.p5]     OK
[UTest::test.affine.wfapt0] OK
[UTest::test.affine.wfapt1] OK
[UTest::test.affine2p]      OK
[UTest::test.biwfa.affine]  OK
[UTest::test.biwfa.affine.p0]   OK
[UTest::test.biwfa.affine.p1]   OK
[UTest::test.biwfa.affine.p2]   OK
[UTest::test.biwfa.affine.p3]   OK
[UTest::test.biwfa.affine.p4]   OK
[UTest::test.biwfa.affine.p5]   OK
[UTest::test.biwfa.affine.wfapt0]   OK
[UTest::test.biwfa.affine.wfapt1]   OK
[UTest::test.biwfa.affine2p]    OK
[UTest::test.biwfa.edit]    OK
[UTest::test.biwfa.indel]   OK
[UTest::test.biwfa.score.affine]    OK
[UTest::test.biwfa.score.affine.p0] OK
[UTest::test.biwfa.score.affine.p1] OK
[UTest::test.biwfa.score.affine.p2] OK
[UTest::test.biwfa.score.affine.p3] OK
[UTest::test.biwfa.score.affine.p4] OK
[UTest::test.biwfa.score.affine.p5] OK
[UTest::test.biwfa.score.affine.wfapt0] OK
[UTest::test.biwfa.score.affine.wfapt1] OK
[UTest::test.biwfa.score.affine2p]  OK
[UTest::test.biwfa.score.edit]  OK
[UTest::test.biwfa.score.indel] OK
[UTest::test.edit]  OK
[UTest::test.indel]     OK
[UTest::test.pb.affine]     Error
[UTest::test.pb.affine.p0]  Error
[UTest::test.pb.affine.p1]  Error
[UTest::test.pb.affine.p2]  Error
[UTest::test.pb.affine.p3]  Error
[UTest::test.pb.affine.p4]  Error
[UTest::test.pb.affine.p5]  Error
[UTest::test.pb.affine.wfapt0]  Error
[UTest::test.pb.affine.wfapt1]  Error
[UTest::test.pb.affine2p]   Error
[UTest::test.pb.edit]       ok
[UTest::test.pb.indel]      ok
[UTest::test.score.affine]  OK
[UTest::test.score.affine.p0]   OK
[UTest::test.score.affine.p1]   OK
[UTest::test.score.affine.p2]   OK
[UTest::test.score.affine.p3]   OK
[UTest::test.score.affine.p4]   OK
[UTest::test.score.affine.p5]   OK
[UTest::test.score.affine.wfapt0]   OK
[UTest::test.score.affine.wfapt1]   OK
[UTest::test.score.affine2p]    OK
[UTest::test.score.edit]    OK
[UTest::test.score.indel]   OK
>>> Correct: ExitStatus() Signal() Correct(     24 (100.00%)      12 (6.23 )
>>>
>>> ERROR
>>>

I have attached the test results to the launchpad bug

smarco commented 1 year ago

Ok, this looks good.

The failing tests are those using --wfa-memory-mode equal to med or low. These modes trigger the WFA-piggyback implementation and are now deprecated in favour of the BiWFA (ultralow). Thus, in the upcoming updates, they will be removed. There is no need to keep on maintaining them, as BiWFA works consistently better in all cases.

Therefore, avoid using med/low memory modes in the BIG-ENDIAN and you are good to go. In any case, I always suggest using the classic WFA or the memory-efficient BiWFA.

I hope this is satisfactory to you.