bitdefender / bddisasm

bddisasm is a fast, lightweight, x86/x64 instruction decoder. The project also features a fast, basic, x86/x64 instruction emulator, designed specifically to detect shellcode-like behavior.
Apache License 2.0
887 stars 115 forks source link

Decoder failures #4

Closed 0xd4d closed 4 years ago

0xd4d commented 4 years ago

Some bugs I found when testing valid instructions

+ green is bddisasm

64-bit code

-2648059cc7ade4 add rax, 0xffffffffe4adc79c
+2648059cc7ade4 add rax, 0xe4adc79c
-2e6a80 push 0xffffffffffffff80
+2e6a80 push 0x80
-662e6a80 push 0xff80
+662e6a80 push 0x80
-6626c7f8ff7f xbegin 0x800000000002d877
+6626c7f8ff7f xbegin 0xd877
-26c7f800000000 xbegin 0x8000000000025b1a
+26c7f800000000 xbegin 0x25b1a
-dcd1 fcom st0, st1
+dcd1 fcom st1, st0
-dcd9 fcomp st0, st1
+dcd9 fcomp st1, st0
-ddd1 fst st1, st0
+ddd1 fst st0, st1
-ddd9 fstp st1, st0
+ddd9 fstp st0, st1
-660f01d9 vmmcall
+66 db 0x66 (0x80000002)
+0f01d9 vmmcall
-660f09 wbinvd
+66 db 0x66 (0x80000002)
+0f09 wbinvd
-f20f09 wbinvd
+f2 db 0xf2 (0x80000002)
+0f09 wbinvd
-260f12c1 movhlps xmm0, xmm1
+260f12c1 movlps xmm0, xmm1
-6700059cc7ade4 add byte ptr [0xe4b983cd], al
+6700059cc7ade4 add byte ptr [rel 0x7fffffffe4b983cd], al
-67480f1a44257f bndldx bnd0, [rbp+0x7f]
+67480f1a44257f bndldx bnd0, [ebp+0x7f]
-26480fae0448 fxsave64 [rax+rcx*2]
+26480fae0448 fxsave [rax+rcx*2]
-26480fae0c48 fxrstor64 [rax+rcx*2]
+26480fae0c48 fxrstor [rax+rcx*2]
-26f30fe60448 cvtdq2pd xmm0, qword ptr [rax+rcx*2]
+26f30fe60448 cvtdq2pd xmm0, xmmword ptr [rax+rcx*2]
-260f0fc196 pfrcp mm0, mm1
+260f0fc196 pfmin mm0, mm1
-260f0f044896 pfrcp mm0, qword ptr [rax+rcx*2]
+260f0f044896 pfmin mm0, qword ptr [rax+rcx*2]
-26c5f0c2c200 vcmpps xmm0, xmm1, xmm2, 0x0
+26c5f0c2c200 vcmpss xmm0, xmm1, xmm2, 0x0
-26c5f0c2044800 vcmpps xmm0, xmm1, xmmword ptr [rax+rcx*2], 0x0
+26c5f0c2044800 vcmpss xmm0, xmm1, dword ptr [rax+rcx*2], 0x0
-26c4e1f1c4c06d vpinsrw xmm0, xmm1, eax, 0x6d
-26c4e1f1c404486d vpinsrw xmm0, xmm1, word ptr [rax+rcx*2], 0x6d
-26c4e1f9c5c080 vpextrw eax, xmm0, 0x80
-26c4e2759fc2 vfnmsub132ss xmm0, xmm1, xmm2
-26c4e2f59fc2 vfnmsub132sd xmm0, xmm1, xmm2
-26c4e275a9c2 vfmadd213ss xmm0, xmm1, xmm2
-26c4e2f5a9c2 vfmadd213sd xmm0, xmm1, xmm2
-26c4e275abc2 vfmsub213ss xmm0, xmm1, xmm2
-26c4e2f5abc2 vfmsub213sd xmm0, xmm1, xmm2
-26c4e275adc2 vfnmadd213ss xmm0, xmm1, xmm2
-26c4e2f5adc2 vfnmadd213sd xmm0, xmm1, xmm2
-26c4e275afc2 vfnmsub213ss xmm0, xmm1, xmm2
-26c4e2f5afc2 vfnmsub213sd xmm0, xmm1, xmm2
-26c4e275b9c2 vfmadd231ss xmm0, xmm1, xmm2
-26c4e2f5b9c2 vfmadd231sd xmm0, xmm1, xmm2
-26c4e275bbc2 vfmsub231ss xmm0, xmm1, xmm2
-26c4e2f5bbc2 vfmsub231sd xmm0, xmm1, xmm2
-26c4e275bdc2 vfnmadd231ss xmm0, xmm1, xmm2
-26c4e2f5bdc2 vfnmadd231sd xmm0, xmm1, xmm2
-26c4e275bfc2 vfnmsub231ss xmm0, xmm1, xmm2
-26c4e2f5bfc2 vfnmsub231sd xmm0, xmm1, xmm2
-26c4e3f914c0e4 vpextrb eax, xmm0, 0xe4
-26c4e3f915c06d vpextrw eax, xmm0, 0x6d
-26c4e37544c27f vpclmulqdq ymm0, ymm1, ymm2, 0x7f
-26c4e37148c230 vpermil2ps xmm0, xmm1, xmm2, xmm3, 0x0
+26c4e37148c2302e vpermilzz2ps xmm0, xmm1, xmm2, xmm3, 0x2e
-26c4e37149c230 vpermil2pd xmm0, xmm1, xmm2, xmm3, 0x0
+26c4e37149c2304f vpermilzz2pd xmm0, xmm1, xmm2, xmm3, 0x4f
-26c4e37148044820 vpermil2ps xmm0, xmm1, xmmword ptr [rax+rcx*2], xmm2, 0x0
+26c4e371480448202e vpermilzz2ps xmm0, xmm1, xmmword ptr [rax+rcx*2], xmm2, 0x2e
-c4e37149044820 vpermil2pd xmm0, xmm1, xmmword ptr [rax+rcx*2], xmm2, 0x0
+c4e371490448202e vpermilzz2pd xmm0, xmm1, xmmword ptr [rax+rcx*2], xmm2, 0x2e
-c4e3694acb08 vblendvps xmm1, xmm2, xmm3, xmm0
+c4e3694acb08 vblendvps xmm1, xmm2, xmm3, xmm16
-c4e36d4acb08 vblendvps ymm1, ymm2, ymm3, ymm0
+c4e36d4acb08 vblendvps ymm1, ymm2, ymm3, ymm16
-c4e3694bcb08 vblendvpd xmm1, xmm2, xmm3, xmm0
+c4e3694bcb08 vblendvpd xmm1, xmm2, xmm3, xmm16
-c4e36d4bcb08 vblendvpd ymm1, ymm2, ymm3, ymm0
+c4e36d4bcb08 vblendvpd ymm1, ymm2, ymm3, ymm16
-c4e3694ccb08 vpblendvb xmm1, xmm2, xmm3, xmm0
+c4e3694ccb08 vpblendvb xmm1, xmm2, xmm3, xmm16
-c4e36d4ccb08 vpblendvb ymm1, ymm2, ymm3, ymm0
+c4e36d4ccb08 vpblendvb ymm1, ymm2, ymm3, ymm16
-c4e3695ccb08 vfmaddsubps xmm1, xmm2, xmm3, xmm0
+c4e3695ccb08 vfmaddsubps xmm1, xmm2, xmm3, xmm16
-c4e36d5ccb08 vfmaddsubps ymm1, ymm2, ymm3, ymm0
+c4e36d5ccb08 vfmaddsubps ymm1, ymm2, ymm3, ymm16
-c4e3e95ccb08 vfmaddsubps xmm1, xmm2, xmm0, xmm3
+c4e3e95ccb08 vfmaddsubps xmm1, xmm2, xmm16, xmm3
-c4e3ed5ccb08 vfmaddsubps ymm1, ymm2, ymm0, ymm3
+c4e3ed5ccb08 vfmaddsubps ymm1, ymm2, ymm16, ymm3
-c4e3695dcb08 vfmaddsubpd xmm1, xmm2, xmm3, xmm0
+c4e3695dcb08 vfmaddsubpd xmm1, xmm2, xmm3, xmm16
-c4e36d5dcb08 vfmaddsubpd ymm1, ymm2, ymm3, ymm0
+c4e36d5dcb08 vfmaddsubpd ymm1, ymm2, ymm3, ymm16
-c4e3e95dcb08 vfmaddsubpd xmm1, xmm2, xmm0, xmm3
+c4e3e95dcb08 vfmaddsubpd xmm1, xmm2, xmm16, xmm3
-c4e3ed5dcb08 vfmaddsubpd ymm1, ymm2, ymm0, ymm3
+c4e3ed5dcb08 vfmaddsubpd ymm1, ymm2, ymm16, ymm3
-c4e3695ecb08 vfmsubaddps xmm1, xmm2, xmm3, xmm0
+c4e3695ecb08 vfmsubaddps xmm1, xmm2, xmm3, xmm16
-c4e36d5ecb08 vfmsubaddps ymm1, ymm2, ymm3, ymm0
+c4e36d5ecb08 vfmsubaddps ymm1, ymm2, ymm3, ymm16
-c4e3e95ecb08 vfmsubaddps xmm1, xmm2, xmm0, xmm3
+c4e3e95ecb08 vfmsubaddps xmm1, xmm2, xmm16, xmm3
-c4e3ed5ecb08 vfmsubaddps ymm1, ymm2, ymm0, ymm3
+c4e3ed5ecb08 vfmsubaddps ymm1, ymm2, ymm16, ymm3
-c4e3695fcb08 vfmsubaddpd xmm1, xmm2, xmm3, xmm0
+c4e3695fcb08 vfmsubaddpd xmm1, xmm2, xmm3, xmm16
-c4e36d5fcb08 vfmsubaddpd ymm1, ymm2, ymm3, ymm0
+c4e36d5fcb08 vfmsubaddpd ymm1, ymm2, ymm3, ymm16
-c4e3e95fcb08 vfmsubaddpd xmm1, xmm2, xmm0, xmm3
+c4e3e95fcb08 vfmsubaddpd xmm1, xmm2, xmm16, xmm3
-c4e3ed5fcb08 vfmsubaddpd ymm1, ymm2, ymm0, ymm3
+c4e3ed5fcb08 vfmsubaddpd ymm1, ymm2, ymm16, ymm3
-c4e36968cb08 vfmaddps xmm1, xmm2, xmm3, xmm0
+c4e36968cb08 vfmaddps xmm1, xmm2, xmm3, xmm16
-c4e36d68cb08 vfmaddps ymm1, ymm2, ymm3, ymm0
+c4e36d68cb08 vfmaddps ymm1, ymm2, ymm3, ymm16
-c4e3e968cb08 vfmaddps xmm1, xmm2, xmm0, xmm3
+c4e3e968cb08 vfmaddps xmm1, xmm2, xmm16, xmm3
-c4e3ed68cb08 vfmaddps ymm1, ymm2, ymm0, ymm3
+c4e3ed68cb08 vfmaddps ymm1, ymm2, ymm16, ymm3
-c4e36969cb08 vfmaddpd xmm1, xmm2, xmm3, xmm0
+c4e36969cb08 vfmaddpd xmm1, xmm2, xmm3, xmm16
-c4e36d69cb08 vfmaddpd ymm1, ymm2, ymm3, ymm0
+c4e36d69cb08 vfmaddpd ymm1, ymm2, ymm3, ymm16
-c4e3e969cb08 vfmaddpd xmm1, xmm2, xmm0, xmm3
+c4e3e969cb08 vfmaddpd xmm1, xmm2, xmm16, xmm3
-c4e3ed69cb08 vfmaddpd ymm1, ymm2, ymm0, ymm3
+c4e3ed69cb08 vfmaddpd ymm1, ymm2, ymm16, ymm3
-c4e3696acb08 vfmaddss xmm1, xmm2, xmm3, xmm0
+c4e3696acb08 vfmaddss xmm1, xmm2, xmm3, xmm16
-c4e36d6acb08 vfmaddss xmm1, xmm2, xmm3, xmm0
+c4e36d6acb08 vfmaddss xmm1, xmm2, xmm3, xmm16
-c4e3e96acb08 vfmaddss xmm1, xmm2, xmm0, xmm3
+c4e3e96acb08 vfmaddss xmm1, xmm2, xmm16, xmm3
-c4e3ed6acb08 vfmaddss xmm1, xmm2, xmm0, xmm3
+c4e3ed6acb08 vfmaddss xmm1, xmm2, xmm16, xmm3
-c4e3696bcb08 vfmaddsd xmm1, xmm2, xmm3, xmm0
+c4e3696bcb08 vfmaddsd xmm1, xmm2, xmm3, xmm16
-c4e36d6bcb08 vfmaddsd xmm1, xmm2, xmm3, xmm0
+c4e36d6bcb08 vfmaddsd xmm1, xmm2, xmm3, xmm16
-c4e3e96bcb08 vfmaddsd xmm1, xmm2, xmm0, xmm3
+c4e3e96bcb08 vfmaddsd xmm1, xmm2, xmm16, xmm3
-c4e3ed6bcb08 vfmaddsd xmm1, xmm2, xmm0, xmm3
+c4e3ed6bcb08 vfmaddsd xmm1, xmm2, xmm16, xmm3
-c4e3696ccb08 vfmsubps xmm1, xmm2, xmm3, xmm0
+c4e3696ccb08 vfmsubps xmm1, xmm2, xmm3, xmm16
-c4e36d6ccb08 vfmsubps ymm1, ymm2, ymm3, ymm0
+c4e36d6ccb08 vfmsubps ymm1, ymm2, ymm3, ymm16
-c4e3e96ccb08 vfmsubps xmm1, xmm2, xmm0, xmm3
+c4e3e96ccb08 vfmsubps xmm1, xmm2, xmm16, xmm3
-c4e3ed6ccb08 vfmsubps ymm1, ymm2, ymm0, ymm3
+c4e3ed6ccb08 vfmsubps ymm1, ymm2, ymm16, ymm3
-c4e3696dcb08 vfmsubpd xmm1, xmm2, xmm3, xmm0
+c4e3696dcb08 vfmsubpd xmm1, xmm2, xmm3, xmm16
-c4e36d6dcb08 vfmsubpd ymm1, ymm2, ymm3, ymm0
+c4e36d6dcb08 vfmsubpd ymm1, ymm2, ymm3, ymm16
-c4e3e96dcb08 vfmsubpd xmm1, xmm2, xmm0, xmm3
+c4e3e96dcb08 vfmsubpd xmm1, xmm2, xmm16, xmm3
-c4e3ed6dcb08 vfmsubpd ymm1, ymm2, ymm0, ymm3
+c4e3ed6dcb08 vfmsubpd ymm1, ymm2, ymm16, ymm3
-c4e3696ecb08 vfmsubss xmm1, xmm2, xmm3, xmm0
+c4e3696ecb08 vfmsubss xmm1, xmm2, xmm3, xmm16
-c4e36d6ecb08 vfmsubss xmm1, xmm2, xmm3, xmm0
+c4e36d6ecb08 vfmsubss xmm1, xmm2, xmm3, xmm16
-c4e3e96ecb08 vfmsubss xmm1, xmm2, xmm0, xmm3
+c4e3e96ecb08 vfmsubss xmm1, xmm2, xmm16, xmm3
-c4e3ed6ecb08 vfmsubss xmm1, xmm2, xmm0, xmm3
+c4e3ed6ecb08 vfmsubss xmm1, xmm2, xmm16, xmm3
-c4e3696fcb08 vfmsubsd xmm1, xmm2, xmm3, xmm0
+c4e3696fcb08 vfmsubsd xmm1, xmm2, xmm3, xmm16
-c4e36d6fcb08 vfmsubsd xmm1, xmm2, xmm3, xmm0
+c4e36d6fcb08 vfmsubsd xmm1, xmm2, xmm3, xmm16
-c4e3e96fcb08 vfmsubsd xmm1, xmm2, xmm0, xmm3
+c4e3e96fcb08 vfmsubsd xmm1, xmm2, xmm16, xmm3
-c4e3ed6fcb08 vfmsubsd xmm1, xmm2, xmm0, xmm3
+c4e3ed6fcb08 vfmsubsd xmm1, xmm2, xmm16, xmm3
-c4e36978cb08 vfnmaddps xmm1, xmm2, xmm3, xmm0
+c4e36978cb08 vfnmaddps xmm1, xmm2, xmm3, xmm16
-c4e36d78cb08 vfnmaddps ymm1, ymm2, ymm3, ymm0
+c4e36d78cb08 vfnmaddps ymm1, ymm2, ymm3, ymm16
-c4e3e978cb08 vfnmaddps xmm1, xmm2, xmm0, xmm3
+c4e3e978cb08 vfnmaddps xmm1, xmm2, xmm16, xmm3
-c4e3ed78cb08 vfnmaddps ymm1, ymm2, ymm0, ymm3
+c4e3ed78cb08 vfnmaddps ymm1, ymm2, ymm16, ymm3
-c4e36979cb08 vfnmaddpd xmm1, xmm2, xmm3, xmm0
+c4e36979cb08 vfnmaddpd xmm1, xmm2, xmm3, xmm16
-c4e36d79cb08 vfnmaddpd ymm1, ymm2, ymm3, ymm0
+c4e36d79cb08 vfnmaddpd ymm1, ymm2, ymm3, ymm16
-c4e3e979cb08 vfnmaddpd xmm1, xmm2, xmm0, xmm3
+c4e3e979cb08 vfnmaddpd xmm1, xmm2, xmm16, xmm3
-c4e3ed79cb08 vfnmaddpd ymm1, ymm2, ymm0, ymm3
+c4e3ed79cb08 vfnmaddpd ymm1, ymm2, ymm16, ymm3
-c4e3697acb08 vfnmaddss xmm1, xmm2, xmm3, xmm0
+c4e3697acb08 vfnmaddss xmm1, xmm2, xmm3, xmm16
-c4e36d7acb08 vfnmaddss xmm1, xmm2, xmm3, xmm0
+c4e36d7acb08 vfnmaddss xmm1, xmm2, xmm3, xmm16
-c4e3e97acb08 vfnmaddss xmm1, xmm2, xmm0, xmm3
+c4e3e97acb08 vfnmaddss xmm1, xmm2, xmm16, xmm3
-c4e3ed7acb08 vfnmaddss xmm1, xmm2, xmm0, xmm3
+c4e3ed7acb08 vfnmaddss xmm1, xmm2, xmm16, xmm3
-c4e3697bcb08 vfnmaddsd xmm1, xmm2, xmm3, xmm0
+c4e3697bcb08 vfnmaddsd xmm1, xmm2, xmm3, xmm16
-c4e36d7bcb08 vfnmaddsd xmm1, xmm2, xmm3, xmm0
+c4e36d7bcb08 vfnmaddsd xmm1, xmm2, xmm3, xmm16
-c4e3e97bcb08 vfnmaddsd xmm1, xmm2, xmm0, xmm3
+c4e3e97bcb08 vfnmaddsd xmm1, xmm2, xmm16, xmm3
-c4e3ed7bcb08 vfnmaddsd xmm1, xmm2, xmm0, xmm3
+c4e3ed7bcb08 vfnmaddsd xmm1, xmm2, xmm16, xmm3
-c4e3697ccb08 vfnmsubps xmm1, xmm2, xmm3, xmm0
+c4e3697ccb08 vfnmsubps xmm1, xmm2, xmm3, xmm16
-c4e36d7ccb08 vfnmsubps ymm1, ymm2, ymm3, ymm0
+c4e36d7ccb08 vfnmsubps ymm1, ymm2, ymm3, ymm16
-c4e3e97ccb08 vfnmsubps xmm1, xmm2, xmm0, xmm3
+c4e3e97ccb08 vfnmsubps xmm1, xmm2, xmm16, xmm3
-c4e3ed7ccb08 vfnmsubps ymm1, ymm2, ymm0, ymm3
+c4e3ed7ccb08 vfnmsubps ymm1, ymm2, ymm16, ymm3
-c4e3697dcb08 vfnmsubpd xmm1, xmm2, xmm3, xmm0
+c4e3697dcb08 vfnmsubpd xmm1, xmm2, xmm3, xmm16
-c4e36d7dcb08 vfnmsubpd ymm1, ymm2, ymm3, ymm0
+c4e36d7dcb08 vfnmsubpd ymm1, ymm2, ymm3, ymm16
-c4e3e97dcb08 vfnmsubpd xmm1, xmm2, xmm0, xmm3
+c4e3e97dcb08 vfnmsubpd xmm1, xmm2, xmm16, xmm3
-c4e3ed7dcb08 vfnmsubpd ymm1, ymm2, ymm0, ymm3
+c4e3ed7dcb08 vfnmsubpd ymm1, ymm2, ymm16, ymm3
-c4e3697ecb08 vfnmsubss xmm1, xmm2, xmm3, xmm0
+c4e3697ecb08 vfnmsubss xmm1, xmm2, xmm3, xmm16
-c4e36d7ecb08 vfnmsubss xmm1, xmm2, xmm3, xmm0
+c4e36d7ecb08 vfnmsubss xmm1, xmm2, xmm3, xmm16
-c4e3e97ecb08 vfnmsubss xmm1, xmm2, xmm0, xmm3
+c4e3e97ecb08 vfnmsubss xmm1, xmm2, xmm16, xmm3
-c4e3ed7ecb08 vfnmsubss xmm1, xmm2, xmm0, xmm3
+c4e3ed7ecb08 vfnmsubss xmm1, xmm2, xmm16, xmm3
-c4e3697fcb08 vfnmsubsd xmm1, xmm2, xmm3, xmm0
+c4e3697fcb08 vfnmsubsd xmm1, xmm2, xmm3, xmm16
-c4e36d7fcb08 vfnmsubsd xmm1, xmm2, xmm3, xmm0
+c4e36d7fcb08 vfnmsubsd xmm1, xmm2, xmm3, xmm16
-c4e3e97fcb08 vfnmsubsd xmm1, xmm2, xmm0, xmm3
+c4e3e97fcb08 vfnmsubsd xmm1, xmm2, xmm16, xmm3
-c4e3ed7fcb08 vfnmsubsd xmm1, xmm2, xmm0, xmm3
+c4e3ed7fcb08 vfnmsubsd xmm1, xmm2, xmm16, xmm3
-67c4e3f15c0520a4274e6d vfmaddsubps xmm0, xmm1, xmm6, xmmword ptr [0x800000004e8dad49]
+67c4e3f15c0520a4274e6d vfmaddsubps xmm0, xmm1, xmm22, xmmword ptr [0x800000004e8dad49]
-c4e3f15d0520a4274e6d vfmaddsubpd xmm0, xmm1, xmm6, xmmword ptr [0x800000004e8dee7f]
+c4e3f15d0520a4274e6d vfmaddsubpd xmm0, xmm1, xmm22, xmmword ptr [0x800000004e8dee7f]
-67c4e3f55d445a206d vfmaddsubpd ymm0, ymm1, ymm6, ymmword ptr [edx+ebx*2+0x20]
+67c4e3f55d445a206d vfmaddsubpd ymm0, ymm1, ymm22, ymmword ptr [edx+ebx*2+0x20]
-67c4e3f15e04e520ffffff7f vfmsubaddps xmm0, xmm1, xmm7, xmmword ptr [0xffffff20]
+67c4e3f15e04e520ffffff7f vfmsubaddps xmm0, xmm1, xmm23, xmmword ptr [0xffffff20]
-c4e3f55e445a207f vfmsubaddps ymm0, ymm1, ymm7, ymmword ptr [rdx+rbx*2+0x20]
+c4e3f55e445a207f vfmsubaddps ymm0, ymm1, ymm23, ymmword ptr [rdx+rbx*2+0x20]
-67c4c3f15f843520a4274e6d vfmsubaddpd xmm0, xmm1, xmm6, xmmword ptr [r13d+esi+0x4e27a420]
+67c4c3f15f843520a4274e6d vfmsubaddpd xmm0, xmm1, xmm22, xmmword ptr [r13d+esi+0x4e27a420]
-c4c3f55f846520a4274e6d vfmsubaddpd ymm0, ymm1, ymm6, ymmword ptr [r13+0x4e27a420]
+c4c3f55f846520a4274e6d vfmsubaddpd ymm0, ymm1, ymm22, ymmword ptr [r13+0x4e27a420]
-67c4e3f168445a206d vfmaddps xmm0, xmm1, xmm6, xmmword ptr [edx+ebx*2+0x20]
+67c4e3f168445a206d vfmaddps xmm0, xmm1, xmm22, xmmword ptr [edx+ebx*2+0x20]
...etc...
-8fe86885cb08 vpmacssww xmm1, xmm2, xmm3, xmm0
+8fe86885cb08 vpmacssww xmm1, xmm2, xmm3, xmm16
-8fe86886cb08 vpmacsswd xmm1, xmm2, xmm3, xmm0
+8fe86886cb08 vpmacsswd xmm1, xmm2, xmm3, xmm16
-8fe86887cb08 vpmacssdql xmm1, xmm2, xmm3, xmm0
+8fe86887cb08 vpmacssdql xmm1, xmm2, xmm3, xmm16
-8fe8688ecb08 vpmacssdd xmm1, xmm2, xmm3, xmm0
+8fe8688ecb08 vpmacssdd xmm1, xmm2, xmm3, xmm16
-8fe8688fcb08 vpmacssdqh xmm1, xmm2, xmm3, xmm0
+8fe8688fcb08 vpmacssdqh xmm1, xmm2, xmm3, xmm16
-8fe86895cb08 vpmacsww xmm1, xmm2, xmm3, xmm0
+8fe86895cb08 vpmacsww xmm1, xmm2, xmm3, xmm16
-8fe86896cb08 vpmacswd xmm1, xmm2, xmm3, xmm0
+8fe86896cb08 vpmacswd xmm1, xmm2, xmm3, xmm16
-8fe86897cb08 vpmacsdql xmm1, xmm2, xmm3, xmm0
+8fe86897cb08 vpmacsdql xmm1, xmm2, xmm3, xmm16
-8fe8689ecb08 vpmacsdd xmm1, xmm2, xmm3, xmm0
+8fe8689ecb08 vpmacsdd xmm1, xmm2, xmm3, xmm16
-8fe8689fcb08 vpmacsdqh xmm1, xmm2, xmm3, xmm0
+8fe8689fcb08 vpmacsdqh xmm1, xmm2, xmm3, xmm16
-8fe868a2cb08 vpcmov xmm1, xmm2, xmm3, xmm0
+8fe868a2cb08 vpcmov xmm1, xmm2, xmm3, xmm16
-8fe86ca2cb08 vpcmov ymm1, ymm2, ymm3, ymm0
+8fe86ca2cb08 vpcmov ymm1, ymm2, ymm3, ymm16
-8fe8e8a2cb08 vpcmov xmm1, xmm2, xmm0, xmm3
+8fe8e8a2cb08 vpcmov xmm1, xmm2, xmm16, xmm3
-8fe8eca2cb08 vpcmov ymm1, ymm2, ymm0, ymm3
+8fe8eca2cb08 vpcmov ymm1, ymm2, ymm16, ymm3
-8fe868a3cb08 vpperm xmm1, xmm2, xmm3, xmm0
+8fe868a3cb08 vpperm xmm1, xmm2, xmm3, xmm16
-8fe8e8a3cb08 vpperm xmm1, xmm2, xmm0, xmm3
+8fe8e8a3cb08 vpperm xmm1, xmm2, xmm16, xmm3
-8fe868a6cb08 vpmadcsswd xmm1, xmm2, xmm3, xmm0
+8fe868a6cb08 vpmadcsswd xmm1, xmm2, xmm3, xmm16
-8fe868b6cb08 vpmadcswd xmm1, xmm2, xmm3, xmm0
+8fe868b6cb08 vpmadcswd xmm1, xmm2, xmm3, xmm16
-26c4e269900448 vpgatherdd xmm0, dword ptr [rax+xmm1*2], xmm2
+26c4e269900448 vpgatherdd xmm0, xmmword ptr [rax+xmm1*2], xmm2
-26c4e2e9900448 vpgatherdq xmm0, qword ptr [rax+xmm1*2], xmm2
+26c4e2e9900448 vpgatherdq xmm0, xmmword ptr [rax+xmm1*2], xmm2
-26c4e269910448 vpgatherqd xmm0, dword ptr [rax+xmm1*2], xmm2
+26c4e269910448 vpgatherqd xmm0, qword ptr [rax+xmm1*2], xmm2
-26c4e2e9910448 vpgatherqq xmm0, qword ptr [rax+xmm1*2], xmm2
+26c4e2e9910448 vpgatherqq xmm0, xmmword ptr [rax+xmm1*2], xmm2
-26c4e269920448 vgatherdps xmm0, dword ptr [rax+xmm1*2], xmm2
+26c4e269920448 vgatherdps xmm0, xmmword ptr [rax+xmm1*2], xmm2
-26c4e2e9920448 vgatherdpd xmm0, qword ptr [rax+xmm1*2], xmm2
+26c4e2e9920448 vgatherdpd xmm0, xmmword ptr [rax+xmm1*2], xmm2
-26c4e269930448 vgatherqps xmm0, dword ptr [rax+xmm1*2], xmm2
+26c4e269930448 vgatherqps xmm0, qword ptr [rax+xmm1*2], xmm2
-26c4e2e9930448 vgatherqpd xmm0, qword ptr [rax+xmm1*2], xmm2
+26c4e2e9930448 vgatherqpd xmm0, xmmword ptr [rax+xmm1*2], xmm2
-26c4e2759f0448 vfnmsub132ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f59f0448 vfnmsub132sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e275a90448 vfmadd213ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f5a90448 vfmadd213sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e275ab0448 vfmsub213ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f5ab0448 vfmsub213sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e275ad0448 vfnmadd213ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f5ad0448 vfnmadd213sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e275af0448 vfnmsub213ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f5af0448 vfnmsub213sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e275b90448 vfmadd231ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f5b90448 vfmadd231sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e275bb0448 vfmsub231ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f5bb0448 vfmsub231sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e275bd0448 vfnmadd231ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f5bd0448 vfnmadd231sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e275bf0448 vfnmsub231ss xmm0, xmm1, dword ptr [rax+rcx*2]
-26c4e2f5bf0448 vfnmsub231sd xmm0, xmm1, qword ptr [rax+rcx*2]
-26c4e3f9140448e4 vpextrb byte ptr [rax+rcx*2], xmm0, 0xe4
-26c4e3f91504486d vpextrw word ptr [rax+rcx*2], xmm0, 0x6d
-26c4e3754404487f vpclmulqdq ymm0, ymm1, ymmword ptr [rax+rcx*2], 0x7f
-26c4e3f54404487f vpclmulqdq ymm0, ymm1, ymmword ptr [rax+rcx*2], 0x7f
-268fe96895c1 vpshlw xmm0, xmm1, xmm2
+268fe96895c1 vpshlb xmm0, xmm1, xmm2
-268fe96896c1 vpshld xmm0, xmm1, xmm2
+268fe96896c1 vpshlb xmm0, xmm1, xmm2
-62f17e187ac1 vcvtudq2pd zmm0, ymm1
-62f17e997ac1 vcvtudq2pd zmm0{k1}{z}, ymm1
-62f177187bc0 vcvtusi2sd xmm0, xmm1, eax
-62f17e18e6c1 vcvtdq2pd zmm0, ymm1
-62f17e99e6c1 vcvtdq2pd zmm0{k1}{z}, ymm1
-6762f17d2871556d7f vpsrlw ymm0, ymmword ptr [ebp+0xda0], 0x7f
+6762f17d2871556d7f vpsrlw ymm0, ymmword ptr [ebp+0x6d0], 0x7f
-6762f17d4871556d7f vpsrlw zmm0, zmmword ptr [ebp+0x1b40], 0x7f
+6762f17d4871556d7f vpsrlw zmm0, zmmword ptr [ebp+0x6d0], 0x7f
-6762f17d2871656de4 vpsraw ymm0, ymmword ptr [ebp+0xda0], 0xe4
+6762f17d2871656de4 vpsraw ymm0, ymmword ptr [ebp+0x6d0], 0xe4
-6762f17d4871656de4 vpsraw zmm0, zmmword ptr [ebp+0x1b40], 0xe4
+6762f17d4871656de4 vpsraw zmm0, zmmword ptr [ebp+0x6d0], 0xe4
-6762f17d2871756d80 vpsllw ymm0, ymmword ptr [ebp+0xda0], 0x80
+6762f17d2871756d80 vpsllw ymm0, ymmword ptr [ebp+0x6d0], 0x80
-6762f17d4871756d80 vpsllw zmm0, zmmword ptr [ebp+0x1b40], 0x80
+6762f17d4871756d80 vpsllw zmm0, zmmword ptr [ebp+0x6d0], 0x80
-6762f1fd0879456d vcvtpd2uqq xmm0, xmmword ptr [ebp+0x6d0]
+6762f1fd0879456d vcvtpd2uqq xmm0, xmmword ptr [ebp+0x368]
-6762f1fd2879456d vcvtpd2uqq ymm0, ymmword ptr [ebp+0xda0]
+6762f1fd2879456d vcvtpd2uqq ymm0, ymmword ptr [ebp+0x6d0]
-6762f1fd4879456d vcvtpd2uqq zmm0, zmmword ptr [ebp+0x1b40]
+6762f1fd4879456d vcvtpd2uqq zmm0, zmmword ptr [ebp+0xda0]
-6762f17508c445e400 vpinsrw xmm0, xmm1, word ptr [ebp-0x38], 0x0
+6762f17508c445e400 vpinsrw xmm0, xmm1, word ptr [ebp-0x70], 0x0
-6762f27d08624580 vpexpandb xmm0, xmmword ptr [ebp-0x80]
+6762f27d08624580 vpexpandb xmm0, xmmword ptr [ebp-0x200]
-6762f27d28624580 vpexpandb ymm0, ymmword ptr [ebp-0x80]
+6762f27d28624580 vpexpandb ymm0, ymmword ptr [ebp-0x200]
-6762f27d48624580 vpexpandb zmm0, zmmword ptr [ebp-0x80]
+6762f27d48624580 vpexpandb zmm0, zmmword ptr [ebp-0x200]
-6762f2fd08624580 vpexpandw xmm0, xmmword ptr [ebp-0x100]
+6762f2fd08624580 vpexpandw xmm0, xmmword ptr [ebp-0x400]
-6762f2fd28624580 vpexpandw ymm0, ymmword ptr [ebp-0x100]
+6762f2fd28624580 vpexpandw ymm0, ymmword ptr [ebp-0x400]
-6762f2fd48624580 vpexpandw zmm0, zmmword ptr [ebp-0x100]
+6762f2fd48624580 vpexpandw zmm0, zmmword ptr [ebp-0x400]
-6762f27d0863457f vpcompressb xmmword ptr [ebp+0x7f], xmm0
+6762f27d0863457f vpcompressb xmmword ptr [ebp+0x1fc], xmm0
-6762f27d2863457f vpcompressb ymmword ptr [ebp+0x7f], ymm0
+6762f27d2863457f vpcompressb ymmword ptr [ebp+0x1fc], ymm0
-6762f27d4863457f vpcompressb zmmword ptr [ebp+0x7f], zmm0
+6762f27d4863457f vpcompressb zmmword ptr [ebp+0x1fc], zmm0
-6762f2fd0863457f vpcompressw xmmword ptr [ebp+0xfe], xmm0
+6762f2fd0863457f vpcompressw xmmword ptr [ebp+0x3f8], xmm0
-6762f2fd2863457f vpcompressw ymmword ptr [ebp+0xfe], ymm0
+6762f2fd2863457f vpcompressw ymmword ptr [ebp+0x3f8], ymm0
-6762f2fd4863457f vpcompressw zmmword ptr [ebp+0xfe], zmm0
+6762f2fd4863457f vpcompressw zmmword ptr [ebp+0x3f8], zmm0
-62f27e18720448 vcvtneps2bf16 xmm0, dword ptr [rax+rcx*2]{1to4}
-62f27e99720448 vcvtneps2bf16 xmm0{k1}{z}, dword ptr [rax+rcx*2]{1to4}
-62f27e38720448 vcvtneps2bf16 xmm0, dword ptr [rax+rcx*2]{1to8}
-62f27eb9720448 vcvtneps2bf16 xmm0{k1}{z}, dword ptr [rax+rcx*2]{1to8}
-62f27e58720448 vcvtneps2bf16 ymm0, dword ptr [rax+rcx*2]{1to16}
-62f27ed9720448 vcvtneps2bf16 ymm0{k1}{z}, dword ptr [rax+rcx*2]{1to16}
-62f27d09a00440 vpscatterdd dword ptr [rax+xmm0*2], k1, xmm0
-62f27d29a00440 vpscatterdd dword ptr [rax+ymm0*2], k1, ymm0
-62f27d49a00440 vpscatterdd dword ptr [rax+zmm0*2], k1, zmm0
-62f2fd09a00440 vpscatterdq qword ptr [rax+xmm0*2], k1, xmm0
-62f2fd29a00440 vpscatterdq qword ptr [rax+xmm0*2], k1, ymm0
-62f2fd49a00440 vpscatterdq qword ptr [rax+ymm0*2], k1, zmm0
-62f27d09a10440 vpscatterqd dword ptr [rax+xmm0*2], k1, xmm0
-62f27d29a10440 vpscatterqd dword ptr [rax+ymm0*2], k1, xmm0
-62f27d49a10440 vpscatterqd dword ptr [rax+zmm0*2], k1, ymm0
-62f2fd09a10440 vpscatterqq qword ptr [rax+xmm0*2], k1, xmm0
-62f2fd29a10440 vpscatterqq qword ptr [rax+ymm0*2], k1, ymm0
-62f2fd49a10440 vpscatterqq qword ptr [rax+zmm0*2], k1, zmm0
-62f27d09a20440 vscatterdps dword ptr [rax+xmm0*2], k1, xmm0
-62f27d29a20440 vscatterdps dword ptr [rax+ymm0*2], k1, ymm0
-62f27d49a20440 vscatterdps dword ptr [rax+zmm0*2], k1, zmm0
-62f2fd09a20440 vscatterdpd qword ptr [rax+xmm0*2], k1, xmm0
-62f2fd29a20440 vscatterdpd qword ptr [rax+xmm0*2], k1, ymm0
-62f2fd49a20440 vscatterdpd qword ptr [rax+ymm0*2], k1, zmm0
-62f27d09a30440 vscatterqps dword ptr [rax+xmm0*2], k1, xmm0
-62f27d29a30440 vscatterqps dword ptr [rax+ymm0*2], k1, xmm0
-62f27d49a30440 vscatterqps dword ptr [rax+zmm0*2], k1, ymm0
-62f2fd09a30440 vscatterqpd qword ptr [rax+xmm0*2], k1, xmm0
-62f2fd29a30440 vscatterqpd qword ptr [rax+ymm0*2], k1, ymm0
-62f2fd49a30440 vscatterqpd qword ptr [rax+zmm0*2], k1, zmm0
-2662f27d49c60c40 vgatherpf0dps dword ptr [rax+zmm0*2], k1
+2662f27d49c60c40 vgatherpf0dps zmmword ptr [rax+zmm0*2]{k1}
...etc...
-6762f37d081445e47f vpextrb byte ptr [ebp-0x1c], xmm0, 0x7f
+6762f37d081445e47f vpextrb byte ptr [ebp-0x70], xmm0, 0x7f
-6762f3fd081445e47f vpextrb byte ptr [ebp-0x1c], xmm0, 0x7f
+6762f3fd081445e47f vpextrb byte ptr [ebp-0xe0], xmm0, 0x7f
-6762f37d0815456d00 vpextrw word ptr [ebp+0xda], xmm0, 0x0
+6762f37d0815456d00 vpextrw word ptr [ebp+0x1b4], xmm0, 0x0
-6762f3fd0815456d00 vpextrw word ptr [ebp+0xda], xmm0, 0x0
+6762f3fd0815456d00 vpextrw word ptr [ebp+0x368], xmm0, 0x0
-6762f3fd0817457f6d vextractps dword ptr [ebp+0x1fc], xmm0, 0x6d
+6762f3fd0817457f6d vextractps dword ptr [ebp+0x3f8], xmm0, 0x6d
-6762f375082045e4e4 vpinsrb xmm0, xmm1, byte ptr [ebp-0x1c], 0xe4
+6762f375082045e4e4 vpinsrb xmm0, xmm1, byte ptr [ebp-0x70], 0xe4
-6762f3f5082045e4e4 vpinsrb xmm0, xmm1, byte ptr [ebp-0x1c], 0xe4
+6762f3f5082045e4e4 vpinsrb xmm0, xmm1, byte ptr [ebp-0xe0], 0xe4

32-bit code

-2660 pushad
+2660 pusha
-2661 popad
+2661 popa
-2682c0e4 add al, 0xe4
+2682c0e42e82c0 add eax, 0xc0822ee4
-2682c86d or al, 0x6d
+2682c86d2e82c8 or eax, 0xc8822e6d
-2682d080 adc al, 0x80
+2682d0802e82d0 adc eax, 0xd0822e80
-2682d87f sbb al, 0x7f
+2682d87f2e82d8 sbb eax, 0xd8822e7f
-2682e000 and al, 0x0
+2682e0002e82e0 and eax, 0xe0822e00
-2682e8e4 sub al, 0xe4
+2682e8e42e82e8 sub eax, 0xe8822ee4
-2682f06d xor al, 0x6d
+2682f06d2e82f0 xor eax, 0xf0822e6d
-2682f880 cmp al, 0x80
+2682f8802e82f8 cmp eax, 0xf8822e80
-6626c7f8ff7f xbegin 0x8001f1fb
+6626c7f8ff7f xbegin 0xf1fb
-260f05 syscall
+260f05 loadall
-260f07 sysret
+260f07 loadalld
-260f97c0 setnbe al
+260f97c0 setnb al
-f3660fc7f8 rdpid eax
+f3660fc7f8 rdpid ax
-f30f1b05a4274e6d bndmk bnd0, [0x6d4e27a4]
-c4e13a10ca vmovss xmm1, xmm0, xmm2
-c4e13b10ca vmovsd xmm1, xmm0, xmm2
-c4e13812ca vmovhlps xmm1, xmm0, xmm2
...most vex instructions with the same bug in 32-bit mode...

There could be more bugs but there are too many diffs due to the above bug.

vlutas commented 4 years ago

Wow, impressive & valuable work! Do you have a script, or a tool which runs these tests automatically? The issues you identified should be fixed in the latest version. Here are the comments for each identified issue:

64 bit

32 bit

0xd4d commented 4 years ago

FaD. When using the address size override in 64 bit mode, it only demotes the register size from 64 to 32 bits. RIP relative addressing works normally even if 0x67 prefix is used. This is what Xed decodes, and this what the actual hardware does.

A 67 prefix truncates the address to 32 bits in 64-bit mode, so in effect it uses EIP instead of RIP. It's no difference than when 67 selects EBX instead of RBX. The result should be an address with the upper 32 bits cleared. See 2.2.1.6 in SDM vol 2, last section.

Note that according to the SDM, encoding these instructions with embedded rounding is ignored. Although bddisasm did fail to decode them initially, the output you provided also seems to be wrong, as it erroneously promotes the vector length to 512 bits. I don't have hardware to actually run these and see what is actually happening; do you have a supporting CPU?

No I haven't tested real HW, this CPU doesn't have AVX-512 instructions.

The SDM says {er} is ignored (these bits aren't used at all), nothing else seems to change. It has no other bits left to differentiate which instruction to decode (128, 256 or 512 bits) so it must use only one of them which is the 512-bit version, which is also the only one that can use {er}. See also table 2-36 in SDM vol 2.

vlutas commented 4 years ago

A 67 prefix truncates the address to 32 bits in 64-bit mode, so in effect it uses EIP instead of RIP. It's no difference than when 67 selects EBX instead of RBX. The result should be an address with the upper 32 bits cleared. See 2.2.1.6 in SDM vol 2, last section.

Ah, yes, indeed, I see now that I do not truncate the rel value to 32 bit in the textal output. This is an output/text only bug, though. Thanks for insisting on it.

The SDM says {er} is ignored (these bits aren't used at all), nothing else seems to change. It has no other bits left to differentiate which instruction to decode (128, 256 or 512 bits) so it must use only one of them which is the 512-bit version, which is also the only one that can use {er}. See also table 2-36 in SDM vol 2.

The SDM also describes these instructions without er/sae support, so there's no point in interpreting the evex.b bit at all in this case, since it says that it's ignored (in fact, Xed doesn't seem to decode the instructions at all, which right now I'm not sure it's the correct way). My interpretation is that the instruction will be decoded as if the evex.b bit is 0, but the ambiguity around it should be cleared by running it on supporting hardware. As we see it, there are three posibilities:

  1. The instructions should #UD (as Xed interprets it);
  2. The instructions should be decoded as if evex.b is 0 (interpretation 1 from SDM);
  3. Embedded rounding is not used, but decodes as if it is used (interpretation 2 from SDM); I'll try to clarify this some other way.
0xd4d commented 4 years ago

XED doesn't decode those two examples yet because I just reported it. It does support the other 2 (of 4) instructions that ignore {er}. bddisasm fails to decode when LL=3.

LL=0 62E10F182AD3 vcvtsi2sd xmm18, xmm14, ebx
LL=1 62714F302AD3 vcvtsi2sd xmm10, xmm22, ebx
LL=2 62D14F582AD3 vcvtsi2sd xmm2, xmm6, r11d
LL=3 62F14F782AD3 vcvtsi2sd xmm2, xmm6, ebx
LL=0 62E10F187BD3 vcvtusi2sd xmm18, xmm14, ebx
LL=1 62714F307BD3 vcvtusi2sd xmm10, xmm22, ebx
LL=2 62D14F587BD3 vcvtusi2sd xmm2, xmm6, r11d
LL=3 62F14F787BD3 vcvtusi2sd xmm2, xmm6, ebx
vlutas commented 4 years ago

Handling LL=3 for the ER ignored instructions has the interesting (but obvious) side-effect of decoding the instructions you previously mentioned (since the LL field becomes fixed - 128 or 512 bit, depending on vector/tuple). I'll leave it like this for now, at least until I get to run it on an actual CPU, and see what it actually does. Thanks for the reports once again! Waiting for any other feedback you might have on the matter!