search

lifting-bits / remill

Library for lifting machine code to LLVM bitcode
Apache License 2.0
1.27k stars 143 forks source link

Intermittent test failures on Mac - CPUID #466

Open ekilmer opened 3 years ago

ekilmer commented 3 years ago

I've only seen these failures happen in CI for any version of MacOS and with all tested LLVM versions.

Initial notes from @pgoodman :

we'd have to look at what leaf of cpuid that is testing, so we'd have to find in the manual or something what the meaning of those selectors for cpuid are then figure out if it is sensical for them to every produce different values. It seems unusual to get different values in a given run, though it could also be a problem somewhere in how data is copied around, e.g. maybe lifted_state isn't correctly saving rdx, and so it's always zero. Would need to look at __remill_sync_hyper_call in tests/X86/Run.cpp

The following is the output from this test run:

[ RUN      ] GeneralInstrTest/InstrTest.SemanticsMatchNative/517
/Users/runner/work/remill/remill/tests/X86/Run.cpp:779: Failure
Expected equality of these values:
  lifted_state->gpr
    Which is: 272-byte object <00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-01 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 ... 00-00 00-00 00-00 00-00 0B-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 0C-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 06-90 8B-0C 01-00 00-00>
  native_state->gpr
    Which is: 272-byte object <00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-01 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 ... 00-00 00-00 00-00 00-00 0B-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 0C-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 06-90 8B-0C 01-00 00-00>
Lifted GPR differs from native GPR
/Users/runner/work/remill/remill/tests/X86/Run.cpp:788: Failure
Value of: false
  Actual: false
Expected: true
States did not match for CPUID_2 with ARG1=0xb ARG2=0x0 and CF=0 PF=0 AF=1 ZF=1 SF=1 DF=1 OF=0
/Users/runner/work/remill/remill/tests/X86/Run.cpp:795: Failure
Expected equality of these values:
  lifted_state->gpr.rdx.qword
    Which is: 1
  native_state->gpr.rdx.qword
    Which is: 0
E1210 22:49:49.372480 352570880 Run.cpp:869] Bytes at offset 2264 are different
[  FAILED  ] GeneralInstrTest/InstrTest.SemanticsMatchNative/517, where GetParam() = 0x1126e9280 (133 ms)

It isn't always the same test (but there are many similar error messages), so here is different output from this test run:

[ RUN      ] GeneralInstrTest/InstrTest.SemanticsMatchNative/652
/Users/runner/work/remill/remill/tests/X86/Run.cpp:779: Failure
Expected equality of these values:
  lifted_state->gpr
    Which is: 272-byte object <00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-01 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 ... 00-00 00-00 00-00 00-00 0B-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 0C-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 06-50 95-10 01-00 00-00>
  native_state->gpr
    Which is: 272-byte object <00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-01 00-00 00-00 00-00 00-00 00-00 00-00 00-00 02-00 00-00 00-00 00-00 ... 00-00 00-00 00-00 00-00 0B-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 0C-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 06-50 95-10 01-00 00-00>
Lifted GPR differs from native GPR
/Users/runner/work/remill/remill/tests/X86/Run.cpp:788: Failure
Value of: false
  Actual: false
Expected: true
States did not match for CPUID_2 with ARG1=0xb ARG2=0x0 and CF=1 PF=0 AF=0 ZF=1 SF=0 DF=0 OF=0
/Users/runner/work/remill/remill/tests/X86/Run.cpp:795: Failure
Expected equality of these values:
  lifted_state->gpr.rdx.qword
    Which is: 1
  native_state->gpr.rdx.qword
    Which is: 2
E1214 06:22:16.367506 597679552 Run.cpp:869] Bytes at offset 2264 are different
/Users/runner/work/remill/remill/tests/X86/Run.cpp:779: Failure
Expected equality of these values:
  lifted_state->gpr
    Which is: 272-byte object <00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-01 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 ... 00-00 00-00 00-00 00-00 0B-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 0C-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 06-50 95-10 01-00 00-00>
  native_state->gpr
    Which is: 272-byte object <00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-01 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 ... 00-00 00-00 00-00 00-00 0B-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 0C-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 06-50 95-10 01-00 00-00>
Lifted GPR differs from native GPR
/Users/runner/work/remill/remill/tests/X86/Run.cpp:788: Failure
Value of: false
  Actual: false
Expected: true
States did not match for CPUID_2 with ARG1=0xb ARG2=0x0 and CF=1 PF=0 AF=0 ZF=0 SF=1 DF=0 OF=1
/Users/runner/work/remill/remill/tests/X86/Run.cpp:795: Failure
Expected equality of these values:
  lifted_state->gpr.rdx.qword
    Which is: 1
  native_state->gpr.rdx.qword
    Which is: 0
E1214 06:22:16.371927 597679552 Run.cpp:869] Bytes at offset 2264 are different
[  FAILED  ] GeneralInstrTest/InstrTest.SemanticsMatchNative/652, where GetParam() = 0x117045600 (56 ms)