ergonomy-joe / u4-decompiled

Decompiled source code for the PC version of Ultima IV - Quest Of The Avatar
66 stars 15 forks source link

Descripency between avatar.exe binary and code block #15

Closed plaidpants closed 6 months ago

plaidpants commented 8 months ago

This D_2CD4 block does not seem to match what I find in the AVATAR.EXE binary. Is is suppose to match or did you need to adjust it? I'm trying to pull the block directly from the EXE instead of hardcoding it in the source. Any ideas?


/*shops'y positions by town*/
unsigned char * D_2CD4[] = {
    &AVATAR[0x11F7F] /*{0x00,0x00,0x00,0x00,0x00,0x1A,0x00,0x00}*/,/*LB*/

    &AVATAR[0x11F87] /*{0x00,0x00,0x00,0x00,0x00,0x0C,0x00,0x00}*/,/*lycaeum*/
    &AVATAR[0x11F8F] /*{0x00,0x00,0x00,0x00,0x00,0x0F,0x00,0x00}*/,/*empath*/
    &AVATAR[0x11F97] /*{0x00,0x00,0x00,0x00,0x00,0x0C,0x00,0x00}*/,/*serpent*/

    &AVATAR[0x11F9F] /*{0x00,0x00,0x0E,0x00,0x1A,0x1B,0x02,0x00}*/,/*MOONGLOW*/
    &AVATAR[0x11FA7] /*{0x03,0x07,0x06,0x02,0x00,0x1D,0x0C,0x00}*/,/*BRITAIN*/
    &AVATAR[0x11FAF] /*{0x09,0x05,0x00,0x13,0x00,0x06,0x1A,0x00}*/,/*JHELOM*/
    &AVATAR[0x11FB7] /*{0x00,0x00,0x18,0x00,0x00,0x19,0x00,0x00}*/,/*YEW.ULT*/
    &AVATAR[0x11FBF] /*{0x1C,0x00,0x00,0x00,0x00,0x00,0x03,0x00}*/,/*MINOC*/
    &AVATAR[0x11FC7] /*{0x14,0x18,0x00,0x02,0x00,0x00,0x03,0x00}*/,/*TRINSIC*/
    &AVATAR[0x11FCF] /*{0x00,0x00,0x11,0x00,0x04,0x1B,0x0D,0x00}*/,/*SKARA*/    /* 00 00 11 00 04 1B 0D A5 ???*/
    &AVATAR[0x11FD7] /*{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}*/,/*MAGINCIA*/ /* 07 B2 00 0A 00 B0 04 1A */

    &AVATAR[0x11FDC] /*{0x00,0x04,0x1A,0x05,0x07,0x00,0x00,0x00}*/,/*paws*/     /* B0 04 1A 05 07 00 00 00 */
    &AVATAR[0x11FE4] /*{0x0B,0x11,0x00,0x19,0x08,0x00,0x00,0x07}*/,/*den*/
    &AVATAR[0x11FEC] /*{0x14,0x00,0x00,0x16,0x00,0x00,0x15,0x1A}*/,/*vesper*/
    &AVATAR[0x11FF4] /*{0x00,0x00,0x00,0x00,0x00,0x1A,0x00,0x00}*/ /*cove*/

};

plaidpants commented 8 months ago

Looking at the apple 2 version it matches your code. 00 00 11 00 04 1B 0D 00 00 00 00 00 00 00 00 00 00 04 1A 05 07 00 00 00 0B 11 00 19 08 00 00 07 14 00 00 16 00

plaidpants commented 8 months ago

Seems like the DOS .exe has some simple compression for zero byte data, it doesn't seem to save very much space though, I see this now in a few other places. I will just need to work around it I guess.

e.g. /shops indexes/ unsigned char D_5196[16] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 0}; // doesn't match exe, need to declare it here 61 74 69 65 21 0A A5 21 B2 00 0F 00 B0 01 02 00

plaidpants commented 8 months ago

Here is some quick and dirty code that I am using to expand the AVATAR.EXE file.

// load a complete copy of the original EXE here
unsigned char AVATAR_original[98208];
// expand into this buffer
unsigned char AVATAR[98208*2];

    // we will use this buffer to reference strings and other things originally in the EXE instead of embedding them in this code
    // this will allow release of the binary of this library/application on other platforms like oculus quest as we don't
    // need to dynamically load DLL at runtime which android does not allow anymore and this allows us to remove
    // all copyright text and data that was contained in the original EXE leaving just the reverse engineered logic of the game engine
    // in this executable code thus avoiding any copyright entanglements
    if (Load("AVATAR.EXE", sizeof(AVATAR_original), &(AVATAR_original)) == -1)
        exit(3);

    // The compiler for the AVATAR.EXE uses simple compression of any block of zeros more than 8 bytes, it is not very efficient,
    // need to expand these zero sections and copy data into a new buffer so strings aren't missing zero delimiters
    // and data blocks that have zeros are properly initialized
    int expanded_index = 0;
    int offset_accumulated = 0;
    for (int index = 0; index < 98208;)
    {
        // we found a marker
        if ((index < 98208 - 7) && // only check up to the end where a marker would fit
            // First 2 bytes are some kind of checksum I think, just ignore for now although we could expand something that is not meant to be expanded with zeros
            (AVATAR_original[index + 2] == (unsigned char)0xB2) && // marker 0xB2 start
            (AVATAR_original[index + 5] == (unsigned char)0x00) && // marker 0x00 ??? this one may be a high byte of the length but for our purposes we can assume it is always zero
            (AVATAR_original[index + 6] == (unsigned char)0xB0))   // marker 0xB0 end
        {
            // get the length of the zero section
            int length = AVATAR_original[index + 3] * 0x100 + AVATAR_original[index + 4];

            // gather some data
            offset_accumulated += length - 7;
            //printf("%08x: %04x \n", index, offset_accumulated);

            // fill with zeros
            for (int j = 0; j < length; j++)
            {
                AVATAR[expanded_index++] = 0x00;
            }

            // skip over marker
            index += 7;

            // continue on
            continue;
        }

        // copy normally
        AVATAR[expanded_index++] = AVATAR_original[index++];
    }

and here is a function to return an updated address given an old address

// Define the offsets
struct Offset {
    size_t address;
    size_t offset;
};

struct Offset offsets[] = {
    {0x0000f2be, 0x0005},
    {0x00010920, 0x0007},
    {0x00010b81, 0x200f},
    {0x0001182a, 0x2017},
    {0x00011fd6, 0x201a},
    {0x0001223a, 0x2623},
    {0x00012251, 0x2925},
    {0x0001225f, 0x2e2e},
    {0x0001440f, 0x2e36},
    {0x000160c1, 0x3837},
    {0x00017806, 0x3838},
    {0x0001780f, 0x387d},
    {0x00017824, 0x388e},
    {0x00017830, 0x38b6},
    {0, 0} // Sentinel value to mark end of array
};

char* getAVATARaddress(size_t originalAddress) 
{
    size_t adjustedAddress = originalAddress;
    for (int i = 0; offsets[i].address != 0; i++) 
    {
        if (originalAddress >= offsets[i].address)
        {
            adjustedAddress += offsets[i].offset;
        }
        else 
        {
            break;
        }
    }
    return &AVATAR[adjustedAddress];
}
ergonomy-joe commented 6 months ago

Hello @plaidpants, sorry for the delay. Well it looks like you found the reason already. ;-)

Tu sum it up, the retail version of AVATAR.EXE is compressed by EXEPACK. This is an option of the Microsoft's linker (there is also a standalone version of the compression tool).

There is a tool for msdos called unpack which allows to inflate the executable. Once you obtain the inflated executable you can find the content of D_2CD4 starting at offset 0x11dd4 (it may vary according to the size of the exe's header).

Does this answer your issue ?

plaidpants commented 6 months ago

I think I have the inflating code working well enough in my code now but I will see if I can find details on the above compression tools to see if I missed anything. Thanks.