Beatdown - a curious problem

[ghotik]

This game (uploaded on MyAbandonware) brought a curious problem. The game blits some graphic that need transparency handling. But, unlike ordinary games, the transparent key color is not hardcoded, the game assumes that for each piece of graphic (held in a 8 bit palettized surface) the pixel at 0,0 is transparent and this is the color to be set on the surface SetColorKey operation. This proceeds more or less like this: 1) lock the surface 2) GetDC to get a GDI handle 3) use GetPixel(0,0) to get the pixel color 4) SetPixel(0,0, color) - the reason for this is unclear ... 5) unlock the surface 6) SetColorKey using the color as low and high color for transparency

All this works perfectly in native mode, but when hooking with DxWnd I get a strange behavior: the GetPixel returns 0xA instead of 0xFC (that corresponds to the blue color of the pixel in the corner) and the transparency is lost ... What could be the reason for all this?

More info here: https://sourceforge.net/p/dxwnd/discussion/general/thread/7ea345ad57/#f47a

[narzoul]

I checked the log snippet you posted in the DxWnd thread, but I don't see any GetPixel calls returning 0xA. Am I missing something? Where is it?

[ghotik]

I forgot to set the flag that enables the GDI scaling and the GetPixel log. I'm posting a good log file here ... But I want to add an important consideration: looking at the log in some previous experiment I saw the GetPixel(0,0) returning either 0xA or 0xFC and both avlues triggers a transparency effect, though not in the correct spots. I have now this idea that because of the DxWnd windowing effect, maybe the GetPixel returns a color that doesn't correspond to 0,0 position but something remapped on the window surface .... I'll try to explain myself better, I got this idea just now ... dxwnd.log I see now that the GetPixel is returning different values again ... this probably depends on the coordinate shift, not the returned values are 0x0 or 0x1 ... see next post

[ghotik]

It is interesting to note that (despite what GetPixel says) the value that fixes the transparency is not the deep blue 0x00FF0000 that I see (and pick) on the screen, but 0xFC that is the index of the color palette that points to the blue.

[narzoul]

I still don't see the connection between GetPixel and SetColorKey.

I checked what happens with Rohitab API Monitor, and the very first GetPixel call for example returns 0x6b6b6b, but SetColorKey still sets 0xFC. Most other GetPixel calls return 0xFF0000, but there is one more odd one that returns 0xa54821, and still the color key is set to 0xFC every time.

Are you sure there is a connection? Have you tried forcing the GetPixel return value to 0xFF0000?

It is interesting to note that (despite what GetPixel says) the value that fixes the transparency is not the deep blue 0x00FF0000 that I see (and pick) on the screen, but 0xFC that is the index of the color palette that points to the blue.

GetPixel should always return RGB 888 values (according to the Windows SDK), but SetColorKey expects the color key value in native pixel format, i.e. in 8-bit palette index in this case.

[ghotik]

Are you sure there is a connection?

Well, the fact is that all this happens inside the same assembly routine, though the ling between 0xFF0000 and 0xFC doesn't seem so clear. For sure, if I force SetColorKey({0xFC, 0xFC}) the result is perfect, but only in this screen where the images have a blue pixel on the left-upper-most pixel 0,0. On other screens the colorkey value should be different. This is the picture with forced 0xFC. In effect I can't get the same result even if I force GetPixel to return 0xFC or 0xFF0000 or 0xFF ... strange? I have to check one thing though .... maybe the game doesn't use the value returned by GetPixel but SetPixel instead?

[narzoul]

Yes, that seems to make more sense. There are 2 SetPixel calls per loop. The first one is always called with 0xFF0000 and returns the same color (note: I'm talking about the main menu only, haven't checked the game any further). The second call is made with the same value that was returned by GetPixel earlier, and returns the same color. So it could be using the return value of the first SetPixel call, mapped to the closest matching palette index.

In effect, I suppose the second call is only made to restore the original pixel color, after overriding it with the transparent color. But why it's not simply using a single GetPixel call, I have no idea. Actually, it seems the transparent color is already determined even before the GetPixel call, so even that shouldn't be needed.

Edit: Maybe it's using SetPixel to find the closest matching color in the palette. Then, it only has to find an exact match in the palette, rather then a nearest match. Still a strange approach, especially when functions like GetNearestPaletteIndex exist.

[ghotik]

You were absolutely right. In this game the transparent color is always the pure blue RGB=0,0,255 but the game uses different palettes, so the purpose of the assembly is exactly the same as GetNearestPaletteIndex, to find where the color is! The problem with DxWnd is that the surfaces and palettes are virtualized and for some reason (yet to be investigated) the GetPixel routine doesn't return an accurate number, so the transparency is not correct. I built a custom GetNearestPaletteIndex subroutine that finds the palette index of the blue in the current palette and returns that value to SetColorKey in the SetColorKey wrapper and all works perfectly. Now the hard task to generalize the idea, but at least now I know what's going on. Thank you very much again.