REP prefix ignored with multi-byte NOP

[GoogleCodeExporter]

In what mode did you try to disassemble (16/32/64)?
64 bits

What is the input buffer (binary stream) you used to reproduce the problem?
Use hex.
'\xf3\x0f\x1f\x90\x90\x90\x90\x90'

What is the expected output (or what instruction)?
rep(z) nop DWORD PTR [rax-0x6f6f6f70]

Which tool did you use to see the expected output?
GNU objdump 2.22 (from GNU Binutils for Ubuntu)

What do you see instead?
NOP DWORD [RAX-0x6f6f6f70]

What version of diStorm are you using? On what platform (Python/EXE/other)?
distorm 3.3 on Python for AMD64, Windows 7

Please provide any additional information below.
N/A

Original issue reported on code.google.com by Electric...@gmail.com on 20 Oct 2012 at 2:09

[GoogleCodeExporter]

I will check it out, it doesn't seem to make much sense in the first place.
Thanks

Original comment by distorm@gmail.com on 21 Oct 2012 at 12:34

[GoogleCodeExporter]

I don't see any reference of an allowed behavior in the Intel Developer's 
Manual.
Can you give me an official documentation ?

Original comment by distorm@gmail.com on 17 Nov 2012 at 12:43

[GoogleCodeExporter]

You are right, in 2.1.2 of the instruction set reference it reads: "Use of 
repeat prefixes and/or undefined opcodes with other Intel 64 or IA-32 
instructions is reserved; such use may cause unpredictable behavior". Still the 
0xf3 byte remains ignored, in the sense that a) it does not appear as a 
modifier in the disassembly of the NOP (yet appears in the decoded 
instruction), and naturally b) does not appear as an non-decoded byte.

Original comment by Electric...@gmail.com on 17 Nov 2012 at 2:24

[GoogleCodeExporter]

It will be flagged as an unused prefix... 
I'm not following you by how you would like to see the behavior of diStorm 
changed?

Original comment by distorm@gmail.com on 18 Nov 2012 at 2:56

[GoogleCodeExporter]

I think being explicit and including the rep prefix in the disassembly, like 
objdump does, would be the better choice. It might pose some advantage when 
reviewing code (for instance, I spotted this 'rep(z) nop DWORD PTR 
[rax-0x6f6f6f70]' with code that was trying to be smart and construct a 
multibyte pause equivalent).

Would you agree?

Original comment by Electric...@gmail.com on 19 Nov 2012 at 12:18

[GoogleCodeExporter]

Unfortunately, I wouldn't agree. If people want to detect malformed code, they 
will have to see that the prefix is dropped (not used). Like what you get from 
printing the hex of the decoded instruction, but it takes someone who 
understands x86 better. Or they might add some functionality like this easily 
on their own to note that the prefix was dropped. Anyway, the way I see it, 
it's not up to diStorm.

Thanks.

Original comment by distorm@gmail.com on 19 Nov 2012 at 5:47

• Changed state: Invalid