search

rapidfuzz / rapidfuzz-cpp

Rapid fuzzy string matching in C++ using the Levenshtein Distance
https://rapidfuzz.github.io/rapidfuzz-cpp
MIT License
248 stars 39 forks source link

Tests fail on i686 architecture #115

Closed hrnciar closed 1 month ago

hrnciar commented 1 month ago

Hello,

I am trying to update rapidfuzz-cpp in Fedora to the latest 3.0.5 version, but I am getting a test failures on i686 architecture.

+ for fuzz_exe in fuzzing/fuzz_*
+ ./fuzzing/fuzz_levenshtein_distance -runs=1000 -max_len=256
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3292292865
INFO: Loaded 1 modules   (1600 inline 8-bit counters): 1600 [0x567779b4, 0x56777ff4), 
INFO: Loaded 1 PC tables (1600 PCs): 1600 [0x56777ff4,0x5677b1f4), 
INFO: A corpus is not provided, starting from an empty corpus
#2  INITED cov: 7 ft: 7 corp: 1/1b exec/s: 0 rss: 32Mb
#221    NEW    cov: 8 ft: 8 corp: 2/6b lim: 6 exec/s: 0 rss: 33Mb L: 5/5 MS: 4 ShuffleBytes-ShuffleBytes-ShuffleBytes-InsertRepeatedBytes-
    NEW_FUNC[1/7]: 0x566e9d30  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x128d30) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    NEW_FUNC[2/7]: 0x566ebc50  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x12ac50) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
#636    NEW    cov: 57 ft: 58 corp: 3/13b lim: 8 exec/s: 0 rss: 34Mb L: 7/7 MS: 5 ChangeBit-InsertRepeatedBytes-ChangeByte-ShuffleBytes-CopyPart-
#648    NEW    cov: 57 ft: 60 corp: 4/21b lim: 8 exec/s: 0 rss: 35Mb L: 8/8 MS: 2 EraseBytes-InsertRepeatedBytes-
#654    NEW    cov: 57 ft: 62 corp: 5/27b lim: 8 exec/s: 0 rss: 35Mb L: 6/8 MS: 1 EraseBytes-
#677    NEW    cov: 59 ft: 73 corp: 6/32b lim: 8 exec/s: 0 rss: 35Mb L: 5/8 MS: 3 ChangeBit-ChangeBinInt-EraseBytes-
s1:  len: 2 content: 0 0 
s2:  len: 2 content: 0 0 
terminate called after throwing an instance of 'std::logic_error'
  what():  levenshtein distance using simd failed (reference_score = 0, score = 2, i = 0)
==425== ERROR: libFuzzer: deadly signal
    #0 0x566ac421  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0xeb421) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #1 0x565feec9  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x3dec9) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #2 0x565e1a01  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x20a01) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #3 0x565e19a3  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x209a3) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #4 0x565ff98a  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x3e98a) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #5 0xf7ede5af  (linux-gate.so.1+0x5af) (BuildId: f400977311e4bf4204e0559cb770f7e0062759d7)
    #6 0xf7ede588  (linux-gate.so.1+0x588) (BuildId: f400977311e4bf4204e0559cb770f7e0062759d7)
    #7 0xf79bf49e  (/lib/libc.so.6+0x7449e) (BuildId: b922c1da65292862a0ca29d0ba28b60115998993)
    #8 0xf79658a0  (/lib/libc.so.6+0x1a8a0) (BuildId: b922c1da65292862a0ca29d0ba28b60115998993)
    #9 0xf794ce96  (/lib/libc.so.6+0x1e96) (BuildId: b922c1da65292862a0ca29d0ba28b60115998993)
    #10 0xf7c6dd0b  (/lib/libstdc++.so.6+0x5d0b) (BuildId: 063d9b89e931c456b94d81b602cfcc759501182d)
    #11 0xf7c85167  (/lib/libstdc++.so.6+0x1d167) (BuildId: 063d9b89e931c456b94d81b602cfcc759501182d)
    #12 0xf7c6d7f1  (/lib/libstdc++.so.6+0x57f1) (BuildId: 063d9b89e931c456b94d81b602cfcc759501182d)
    #13 0xf7c854af  (/lib/libstdc++.so.6+0x1d4af) (BuildId: 063d9b89e931c456b94d81b602cfcc759501182d)
    #14 0x566840af  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0xc30af) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #15 0x566f0755  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x12f755) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #16 0x566ea00e  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x12900e) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #17 0x566f3778  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x132778) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #18 0x565e36f0  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x226f0) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #19 0x565e2780  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x21780) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #20 0x565e4650  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x23650) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #21 0x565e55f4  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x245f4) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #22 0x565cfbf4  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0xebf4) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #23 0x565ffd8c  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x3ed8c) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
    #24 0xf794ee82  (/lib/libc.so.6+0x3e82) (BuildId: b922c1da65292862a0ca29d0ba28b60115998993)
    #25 0xf794ef47  (/lib/libc.so.6+0x3f47) (BuildId: b922c1da65292862a0ca29d0ba28b60115998993)
    #26 0x565c2b56  (/builddir/build/BUILD/rapidfuzz-cpp-3.0.5-build/rapidfuzz-cpp-3.0.5/redhat-linux-build/fuzzing/fuzz_levenshtein_distance+0x1b56) (BuildId: f26904e8541e55d72d2e0d6751ee2de6700e0156)
NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 3 CopyPart-ShuffleBytes-ChangeBinInt-; base unit: 05fe405753166f125559e7c9ac558654f107c7e9
0x2,0x0,0x0,0x0,0x0,0x0,0x0,0x0,
\002\000\000\000\000\000\000\000
artifact_prefix='./'; Test unit written to ./crash-b0aa4b549f325cca9c9dfa6ce1bd6072aeaeac71
Base64: AgAAAAAAAAA=
error: Bad exit status from /var/tmp/rpm-tmp.MovkSF (%check)

Full log: https://kojipkgs.fedoraproject.org//work/tasks/5616/124845616/build.log

Thank you, for the help.

maxbachmann commented 1 month ago

Not really sure what's causing this. Does i686 on Fedora implicate sse2 support?

On my x64 machine I appear to get correct results even when forcing sse2 instead of avx2.

hrnciar commented 1 month ago

You can find the HW details here: https://kojipkgs.fedoraproject.org//work/tasks/5850/124845850/hw_info.log Seems like it uses both sse2 and avx2.

Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat vnmi umip pku ospke avx512_vnni md_clear flush_l1d arch_capabilities
maxbachmann commented 1 month ago

I managed to reproduce this when building a 32 bit binary of the library on my system. Looking into it

maxbachmann commented 1 month ago

I published v3.1.1 which should fix these issues. In addition I added a 32bit ci test to avoid regressions on this.