facebookexperimental / hermit

Hermit launches linux x86_64 programs in a special, hermetically isolated sandbox to control their execution. Hermit translates normal, nondeterministic behavior, into deterministic, repeatable behavior. This can be used for various applications, including replay-debugging, reproducible artifacts, chaos mode concurrency testing and bug analysis.
Other
1.19k stars 31 forks source link

Cargo test failed on two cases #24

Open TianlongLiang opened 1 year ago

TianlongLiang commented 1 year ago

Describe the bug A clear and concise description of what the bug is:

When I run cargo test two test case failed.

test getrandom_intercepted ... FAILED
test rdrand_rdseed_is_masked ... FAILED

failures:

---- getrandom_intercepted stdout ----
Expecting determinism:
Test Run 1:
thread 'getrandom_intercepted' panicked at 'vdso symbol __vdso_clock_gettime's real size is 5 bytes, but trying to replace it with 8 bytes', /home/tl/.cargo/git/checkouts/reverie-9a587e40a0d7d3be/c448d10/reverie-ptrace/src/vdso.rs:148:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- rdrand_rdseed_is_masked stdout ----
Expecting determinism:
Test Run 1:
thread 'rdrand_rdseed_is_masked' panicked at 'Once instance has previously been poisoned', /home/tl/.cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.4.0/src/inline_lazy.rs:30:16

failures:
    getrandom_intercepted
    rdrand_rdseed_is_masked

Indicate any of these common scenarios that apply:

To Reproduce Minimal input to reproduce the behavior.

cargo test

Expected behavior A clear and concise description of what you expected to happen.

all tests pass

Environment

Additional context Attach the logs to this issue as a text file generated by hermit --log=trace --log-file=FOO run.

Add any other context about the problem here. I saw people have similar situation that would explain why I failed the first test. But I am not sure what caused the second test to fail. Is it because of some bug in lazy_static?

TianlongLiang commented 1 year ago

Thanks to the fix mentioned in this issue, the error message I mentioned above is resolved. But a new one emerged:

thread 'rdrand_rdseed_is_masked' panicked at 'Unsupported processor with feature info: FeatureInfo { extended_family_id: 0, extended_model_id: 9, family_id: 6, model_id: 151, stepping_id: 2, brand_index: 0, cflush_cache_line_size: 8, initial_local_apic_id: 4, max_logical_processor_ids: 1, edx_ecx: SSE3 | PCLMULQDQ | SSSE3 | FMA | CMPXCHG16B | PCID | SSE41 | SSE42 | X2APIC | MOVBE | POPCNT | TSC_DEADLINE | AESNI | XSAVE | OSXSAVE | AVX | F16C | RDRAND | HYPERVISOR | FPU | VME | DE | PSE | TSC | MSR | PAE | MCE | CX8 | APIC | SEP | MTRR | PGE | MCA | CMOV | PAT | PSE36 | CLFSH | MMX | FXSR | SSE | SSE2 | SS }

Is this because I tried to run hermit in VM?

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         45 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  12
  On-line CPU(s) list:   0-11
Vendor ID:               GenuineIntel
  Model name:            12th Gen Intel(R) Core(TM) i9-12900KS
    CPU family:          6
    Model:               151
    Thread(s) per core:  1
    Core(s) per socket:  1
    Socket(s):           12
    Stepping:            2
    BogoMIPS:            6835.21
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cm
                         ov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1g
                         b rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopolo
                         gy tsc_reliable nonstop_tsc cpuid tsc_known_freq pni pclmul
                         qdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt t
                         sc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf
                         _lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp i
                         brs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms i
                         nvpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsav
                         ec xgetbv1 xsaves arat umip pku ospke gfni vaes vpclmulqdq 
                         rdpid movdiri movdir64b fsrm md_clear flush_l1d arch_capabi
                         lities
Virtualization features: 
  Hypervisor vendor:     VMware
  Virtualization type:   full
Caches (sum of all):     
  L1d:                   576 KiB (12 instances)
  L1i:                   384 KiB (12 instances)
  L2:                    15 MiB (12 instances)
  L3:                    360 MiB (12 instances)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-11
Vulnerabilities:         
  Itlb multihit:         KVM: Mitigation: VMX unsupported
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl and
                          seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer san
                         itization
  Spectre v2:            Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, P
                         BRSB-eIBRS SW sequence
  Srbds:                 Not affected
  Tsx async abort:       Not affected
jasonwhite commented 1 year ago

You're assumption is correct; we don't support VMs very well at the moment. In particular, we usually can't intercept the CPUID instruction on VMs. We're working on gracefully continuing if we can't intercept the CPUID instruction, but this will mean that Hermit won't be able to determinize the result of CPUID.

TianlongLiang commented 1 year ago

In particular, we usually can't intercept the CPUID instruction on VMs. That's my observation too. At first, the error message displayed is "can't intercept the CPUID" or something. For VMware on Windows, I was able to enable the "virtualized CPU performance counter" feature, which would allow the Linux perf tool to run normally, and it actually can intercept the CPUID. But as you can see from the above message, it seems that the CPU feature list is maybe somehow changed or differed from the normal physical machine. So that it couldn't be handled normally, either