and $0xff, %edi assembles wrong

[bchurchill]

$ ./asm
andl $0xff, %edi
83 e7 ff

but if you objdump this:

0000000000000000 <.data>:
   0:   83 e7 ff                and    $0xffffffff,%edi
   3:   c3                      retq

[bchurchill]

The correct assembly should be

0000000000000000 <.data>:
   0:   81 e7 ff 00 00 00       and    $0xff,%edi
   6:   c3                      retq 

I think I vaguely remember some issue about ambiguity in situations like this. There are two opcodes to choose from, one which does an AND with the register and a sign-extended byte, and the other that does an AND with the register and the given 32-bit immediate. I think for immediates less than 0x7f we should use the byte version and immediates 0x80 and above we should use the 32-bit version. Eric, what are the things I haven't considered?

[bchurchill]

Oh, and for 32-bit immediates between 0xffffffff and 0xffffff80 we should use the 8-bit version, since the sign-extend will work correctly there.

[eschkufz]

Is this a semantic difference, or a distinction without a difference?

My memory from when I tried to get asm to agree with gcc was that everytime you fixed one case like this, some other case would break. I think it's a matter of the logic we use to decide on immediate width being located somewhere fundamentally different than where gcc's is.

On Sun, Feb 22, 2015 at 5:38 PM, Berkeley Churchill < notifications@github.com> wrote:

Oh, and for 32-bit immediates between 0xffffffff and 0xffffff80 we should use the 8-bit version, since the sign-extend will work correctly there.

— Reply to this email directly or view it on GitHub https://github.com/eschkufz/x64asm/issues/158#issuecomment-75478649.

[bchurchill]

This is a semantic difference. I remember us tearing our hair out over this in my rotation. I think the prerequisite to working on this is better testing infrastructure.

[bchurchill]

To be clear, the difference is that if you AND with 0xff, this zeros all but the lower byte; if you AND with 0xffffffff, this is a no-op on a 32-bit register. If you use the short one-byte form, it will always do the latter; if you do the long from, it can do both.

[eschkufz]

Good point. Eech... how to get this right?

On Mon, Feb 23, 2015 at 3:20 PM, Berkeley Churchill < notifications@github.com> wrote:

To be clear, the difference is that if you AND with 0xff, this zeros all but the lower byte; if you AND with 0xffffffff, this is a no-op on a 32-bit register. If you use the short one-byte form, it will always do the latter; if you do the long from, it can do both.

— Reply to this email directly or view it on GitHub https://github.com/eschkufz/x64asm/issues/158#issuecomment-75659920.

[bchurchill]

I think that if the constant, as an unsigned value, is greater than 0x80, or 0x8000 then it needs to be promoted to the next size. I'll try and look at this when I'm in the parser.