autc04 / Retro68

a gcc-based cross-compiler for classic 68K and PPC Macintoshes
GNU General Public License v3.0
548 stars 54 forks source link

ModalDialog() and (short int*) vs. (int*) as *item argument #237

Closed dangu closed 6 months ago

dangu commented 6 months ago

I ran into an interesting effect when trying to use ModalDialog(), using an int as the argument for which item is selected. I made a small test project using three items in a dialog.

This code works:

DialogPtr thedialog;
short itemhitShort=0;

thedialog = GetNewDialog(dialogID,(Ptr)NIL,(WindowPtr)(-1));
ModalDialog(NIL,&itemhitShort);

This code does not work:

DialogPtr thedialog;
int itemhitInt=0;

thedialog = GetNewDialog(dialogID,(Ptr)NIL,(WindowPtr)(-1));
ModalDialog(NIL,(short int*)&itemhitInt);

Using itemhitShort, the returned item numbers are 1,2 or 3 as they should. With itemhitInt, the returned item numbers are 65536, 131072 or 196608

What am I missing here?

(the complete project: https://github.com/dangu/classic_mac/tree/modaldialog)

Edit: Ok, the numbers in hex gives a hint:

0x10000=65536 0x20000=131072 0x30000=196608

Edit 2: I'm using Multiverse and building the project with the latest Retro68 docker image.

autc04 commented 6 months ago

The simple answer is that your first version is correct while your second version is undefined behavior, you're simply not allowed to mix up ints and shorts like that by the C and C++ standards.

In reality however, ModalDialog will just happily write two bytes to the given address.

Other than most (all?) modern platforms, m68K and PowerPC were big endian platforms. That means (short)1 would be 0x00, 0x01, so if you take (int)0 which is 0x00, 0x00, 0x00, 0x00 and overwrite the first two bytes, you get 0x00010000 which is 65536.

On a little-endian platform, you'd get 0x01, 0x00, 0x00, 0x00 which gets interpreted as (int)1, so you can sometimes get away with mixing up your pointer types.

dangu commented 6 months ago

Perfect explanation. Thanks for teaching me C programming basics!