FX3 Halloween Pack - Breaks native grabber, legacyfx3 blurry/missing pixels

[Tempest43528]

Freezy - The new FX3 Halloween pack of Monster Bash and CFTBL broke the native screen grabber code it appears.. Nothing is displaying on DMD now. Everything worked perfect with native until the release of the two new tables this morning.

I tried to switch the legacyfx3 mode and while it does display something on the DMD the DMD is fuzzy/blurry or the DMD is not fully displaying things.

[MikedaSpike]

For me the same. Although --legacyFX3 Also shows a black table (also on old tables).```

C:\Pinball\DMDext>dmdext mirror --source=pinballfx3 --no-virtual
 [1] 2019/10/29 16:07:26.974  INFO | Launching console tool.
 [1] 2019/10/29 16:07:27.036  INFO | PinDMDv1 device not found.
 [1] 2019/10/29 16:07:27.037  WARN | Device PinDMDv1 is not available.
 [1] 2019/10/29 16:07:27.056  INFO | PinDMDv2 device not found.
 [1] 2019/10/29 16:07:27.056  WARN | Device PinDMDv2 is not available.
 [1] 2019/10/29 16:07:27.056  INFO | Checking port COM5 for PinDMDv3...
 [1] 2019/10/29 16:07:27.170  INFO | Found PinDMDv3 device on COM5.
 [1] 2019/10/29 16:07:27.170 DEBUG |    Firmware:    REV-vPin-01008
 [1] 2019/10/29 16:07:27.170 DEBUG |    Resolution:  128x32
 [1] 2019/10/29 16:07:27.273  INFO | Added PinDMDv3 renderer.
 [1] 2019/10/29 16:07:27.273 DEBUG | PIN2DMD device not found.
 [1] 2019/10/29 16:07:27.273  WARN | Device PIN2DMD is not available.
 [1] 2019/10/29 16:07:27.320  INFO | Loading alphanumeric SVGs...
 [1] 2019/10/29 16:07:27.356  INFO | Loading alphanumeric SVGs...
 [1] 2019/10/29 16:07:27.356  INFO | Loading numeric (8) SVGs...
 [1] 2019/10/29 16:07:27.356  INFO | Loading numeric (8) SVGs...
 [1] 2019/10/29 16:07:27.356  INFO | Loading numeric (10) SVGs...
 [1] 2019/10/29 16:07:27.356  INFO | Loading numeric (10) SVGs...
 [1] 2019/10/29 16:07:27.356  INFO | All SVGs loaded.
 [1] 2019/10/29 16:07:27.356  INFO | Added virtual Alphanumeric renderer.
 [1] 2019/10/29 16:07:27.365  INFO | Setting up Render Graph for 2 destination(s)
 [1] 2019/10/29 16:07:27.365  INFO | Connecting Pinball FX3 to PinDMD v3 (ColoredGray2 => ColoredGray2)
 [1] 2019/10/29 16:07:27.381  INFO | Waiting for Pinball FX3 process to start...
 [1] 2019/10/29 16:07:27.405  INFO | Press CTRL+C to close.
 [3] 2019/10/29 16:08:27.477  INFO | Process found, starting capturing...
 [3] 2019/10/29 16:08:27.477  INFO | Reading DMD data from Pinball FX3's memory at 25 fps...
 [3] 2019/10/29 16:08:27.477  INFO | Frames coming in from Pinball FX3.
 [5] 2019/10/29 16:12:05.403  INFO | Terminating DMD data capture from Pinball FX3
 [5] 2019/10/29 16:12:05.403  INFO | Frames stopped from Pinball FX3.

[freezy]

Thanks guys for the heads-up, I've sent a message to NoEx, since it's him maintaining the grabber.

[Tempest43528]

Thanks freezy!

[BlackStar-EoP]

I 'borrowed' your memory offset for my own little DMD hobby project, what I found so far is that the old signature seems to be borked, there are now 4 memory offsets with 10 characters matching from the old signature: uint8_t DMDSIGNATURE[] = { 0x8B, 0x81, 0xFF, 0xFF, 0xFF, 0xFF, 0x89, 0x44, 0x24, 0xFF, 0x8B, 0x81, 0xFF, 0xFF, 0xFF, 0xFF, 0x89, 0x44, 0x24, 0xFF, 0xA1 };

4 matching offsets 0x8B, 0x81, 0x0C, 0x01, 0x00, 0x00, 0x89, 0x44, 0x24, 0x18, 0x8B, 0x81, 0x10, 0x01, 0x00, 0x00, 0x89, 0x44, 0x24, 0x1C, 0x8D

0x8B, 0x81, 0x68, 0x41, 0x00, 0x00, 0x89, 0x44, 0x24, 0x04, 0x8B, 0x81, 0x6C, 0x41, 0x00, 0x00, 0x89, 0x44, 0x24, 0x08, 0x66

0x8B, 0x81, 0x68, 0x41, 0x00, 0x00, 0x89, 0x44, 0x24, 0x20, 0x8B, 0x81, 0x6C, 0x41, 0x00, 0x00, 0x89, 0x44, 0x24, 0x24, 0x8D

0x8B, 0x81, 0xC0, 0x01, 0x00, 0x00, 0x89, 0x44, 0x24, 0x10, 0x8B, 0x81, 0xC4, 0x01, 0x00, 0x00, 0x89, 0x44, 0x24, 0x14, 0xEB

So the final character 0xA1 cannot be found anymore.

I tried to hardcode these 4 offsets, but the first 3 do nothing, and the 4th one seems to crash the entire application. It may be possible that the memory pointers changed.

[freezy]

Cool, let us know if you find out more, and of course, PR welcome :)

[BlackStar-EoP]

I'm going to check a bit deeper, since I now know what kind of values to expect in the 4KB memory block maybe I can backtrack from that offset, but since NoEx is the wizard that found out how to properly do this I expect him to solve it first, if I do manage to find it you bet there will be a PR ;)

[xantari]

It would be awesome if NoEx described his process here for finding the new memory offsets so others may contribute more easily.

[BlackStar-EoP]

Well essentially what I figure out from common sense is this: uint8_t DMDSIGNATURE[] = { 0x8B, 0x81, 0xFF, 0xFF, 0xFF, 0xFF, 0x89, 0x44, 0x24, 0xFF, 0x8B, 0x81, 0xFF, 0xFF, 0xFF, 0xFF, 0x89, 0x44, 0x24, 0xFF, 0xA1 }; all bytes marked as 0xFF are treated as wildcards, so they can be anything, but the rest of the pattern needs to match. At the end of this memory pattern there's a 4 byte pointer. If this pointer is 0, no table is loaded. If it's nonzero, it's pointing to a heap allocated block.

My own function which is basically a copy of the getDMD function inside dmd ext is as follows:

uint32_t FX3Process::get_DMD_ptr() const
{
    uint32_t ptr = 0;
    ReadProcessMemory(m_FX3_process_handle, (void*)(m_DMD_memory_offset), &ptr, sizeof(uint32_t), NULL);
    ReadProcessMemory(m_FX3_process_handle, (void*)(ptr + 0xF0), &ptr, sizeof(uint32_t), NULL);
    ReadProcessMemory(m_FX3_process_handle, (void*)(ptr + 0x34), &ptr, sizeof(uint32_t), NULL);
    return ptr;
}

The pointer is definitely pointing to a class / structure, and at offset 0xF0 there's yet another pointer to yet another structure. I presume this is something like TableDetails / TableResources / DMDInformation. Inside this structure at offset 0x34 there's another pointer and this is the one that is pointing to the DMD memory block. Reading 4KB from this pointer will net you the DMD memory.

Reading the DMD color is very similar:

bool FX3Process::getDMDColor(QColor& color)
{
    uint32_t ptr;
    uint32_t col;

    ReadProcessMemory(m_FX3_process_handle, (void*)(m_DMD_memory_offset), &ptr, sizeof(uint32_t), NULL);
    ReadProcessMemory(m_FX3_process_handle, (void*)(ptr + 0xF0), &ptr, sizeof(uint32_t), NULL);
    ReadProcessMemory(m_FX3_process_handle, (void*)(ptr + 0x50), &ptr, sizeof(uint32_t), NULL);
    ReadProcessMemory(m_FX3_process_handle, (void*)(ptr + 0x08), &col, sizeof(uint32_t), NULL);

    // Color is fetched as ARGB, values seem to be either 0x11 or 0x33 per channel
    uint8_t r = (col >> 16) & 0x000000FF;
    uint8_t g = (col >> 8) & 0x000000FF;
    uint8_t b = col & 0x000000FF;

    if (r == 0 && g == 0 && b == 0)
        return false;

    // 0x33 * 5 = 0xFF
    color = QColor(r * 5, g * 5, b * 5);
    return  true;
}

(Feel free to use my trick for getting the correct color instead of the switch that's currently in the codebase ;) )

I'm currently iterating all the memory blocks that are exactly 4kB in size and try to reverse find the pointer offsets but unfortunately there's 700MB of memory in use and a lot of these values are matching. Basically if you load a Zen Original table, what you are looking for is a memory block of exactly 4kB which contains values of 0, 1 and 2. The Williams tables have very different values for some obscure reason, but if you find the correct block, you should be able to find the address and backtrack from there. Problem is that you have a pointer to the table, (main offset) with a pointer to a sort of dmd structure or something (0xF0) which contains a pointer to the actual DMD memory block (0x34), color is done similar, it takes the same 0xF0 offset to the DMD structure, which has a pointer at offset 0x50, which is likely something like DMDColor information which the color as a 32 bit ARGB int at 0x08.

The biggest problem with all of this is as soon as Zen changes a single member all these offsets are likely to change.

So my assumption is that the internal structure of the classes changed. This could mean that:

• They added/removed members which will move the memory offsets to different locations
• The main pattern changed (I still have no idea what exactly the pattern is, how NoEx found it, why the wildcard bytes are there)

Should be solvable :)

[BlackStar-EoP]

@freezy did you already get a response from NoEx if he's working on it? I tried until the late hours last night but couldn't get any progress, I wanna try something different, I'm going to dump the main memory which should contain this signature in the main menu and when playing a table to find out if the signature became longer and the pointer pointing to the table is now located at a different offset. (should be easily findable since it's 4 0x00 bytes when the table is not loaded and should be valid when it is)

[freezy]

Nope, no reply yet. I'll redirect him to this thread, maybe he's got some pointers!

[BlackStar-EoP]

Ok no worries, I'm on it as well. I have some ideas on how to tackle this and I found my old code from my own DMD widget that allows you to dump the entire process memory (every heap block), could take a while to track down, but shouldn't be impossible. I suspect that the signature we have currently remained largely the same (wishful thinking a bit) but I'll keep you posted :)

[freezy]

If anyone wants to test, the CI's build is here (click on a platform and change to the "artifact" tab).

[Tempest43528]

Thanks Freezy/NoEx - just confirmed I can bring up the new tables using native grabber. Appreciate all your guys work!

[MikedaSpike]

Confirmed it is working for me too. Thanks Freezy, NoEx and everybody who did his investigation to get it working again !

[kds70]

in #197 i told that it´s not working - my mistake, sorry. It´s working ... i forgot to replace the dmdext.exe because i´ve just replaced dmddevide.dll. It works now ... thank you =)