Aspect ratio correction

[kcat]

A quirk of the original game, along with a number of other games from the same era, is that even though they used a 320x200 (16:10) video mode, it was intended for fullscreen display on 4:3 monitors. This resulted in the scene being viewed with non-square/"stretched" pixels. For modern systems which use square-pixel video modes (framebuffer aspect ratio matching the monitor aspect ratio), this requires screen elements to be stretched vertically by a multiplier of 1.2. Both UI elements and the world projection matrix need to include this stretching to appear as they originally did, on top of any letterboxing or pillarboxing.

See here for more information (for the original Doom, but the same applies to Arena as they both used stretched 320x200 on a 4:3 display).

[afritz1]

It shouldn't be too hard for me to correct. SDL is able to stretch surfaces with SDL_BlitScaled(), and that's what I've already been doing when going from 320x200 to the current letterbox resolution. So wherever the letterbox blit dimensions are 320x200, the 200 would be 240 instead.

Maybe it could be in the options, like "AspectCorrection=True/False" or something.

I noticed that DOSBox does this for some games. Usually I just try to find a circle in the game and see if it's squashed or not. That tells me if it's at the intended aspect ratio.

[kcat]

Yeah. I imagine fixing up Renderer::getLetterboxDimensions would handle much of it (BTW, it would probably be better to return an SDL_Rect directly instead of a unique_ptr to one, since a unique_ptr requires an allocation), though it may be necessary to check wherever ORIGINAL_WIDTH or ORIGINAL_HEIGHT are used too.

[afritz1]

I think this change is more involved than I thought. Quite a lot of the interface objects are using fixed offsets from ORIGINAL_HEIGHT instead of screen percentages like ORIGINAL_HEIGHT * 0.75. When I change Renderer::getLetterboxDimensions() to use CORRECTED_HEIGHT of 240, it messes up several of the button positions. I don't think I can do a simple change with SDL_BlitScaled(), either.

On a lighter note, you mentioned that the world projection matrix would be affected as well. It's interesting you say that, because there aren't any transformation matrices being used here (unlike with OpenGL for example). My OpenCL rendering uses the window dimensions instead of the letterbox dimensions anyway, so the 3D scene isn't affected by the corrected aspect ratio. With ray tracing, you get implicit perspective transformation when creating a ray for each pixel. This is one of the things that makes ray tracing more intuitive than rasterization! I suppose one could store the rays in a GPU buffer between frames and run a rotation matrix on them every time the camera rotates if it would noticeably boost performance, but it's often good enough to just recreate them each frame.

[kcat]

I think this change is more involved than I thought. Quite a lot of the interface objects are using fixed offsets from ORIGINAL_HEIGHT instead of screen percentages like ORIGINAL_HEIGHT * 0.75. When I change Renderer::getLetterboxDimensions() to use CORRECTED_HEIGHT of 240, it messes up several of the button positions. I don't think I can do a simple change with SDL_BlitScaled(), either.

Yeah. I think a good way to handle it would be to use normalized coordinates for menu item positions and sizes. Input, e.g. from mouse pointer position or image width/height, needs to be normalized, while output gets denormalized to fit the surface size. In that case, you'd take the aspect ratio correction into account when calculating the normalized image size, and it'll automatically stretch as needed.

My OpenCL rendering uses the window dimensions instead of the letterbox dimensions anyway, so the 3D scene isn't affected by the corrected aspect ratio.

Unfortunately it needs to be. At the end of the day, you're projecting a normalized 3D scene onto a 2D screen, whether you're using polygons or ray-tracing or ray-casting. Without accounting for the non-square pixels the original game had, the view (given the same FOV) will appear vertically squished. Imagine a wall that's 128 units high and 128 units wide. In the original game, when looking at it head-on it would be 20% taller than it is wide, due to the 16:10 buffer it was rendered to getting stretched to 4:3.

[afritz1]

I now see that the designers of DOS games had to make the decision of drawing art for either 320x200 or 320x240 (the documentation says Doom scaled to the latter). Some games might've ended up using different aspect ratios for some of their art assets, though.

I'm slightly confused after reading this comment on a Steam thread. I just looked around in the geoscape and battlescape of X-COM: UFO Defense, and it all seems to have been designed for 16:10, but the person in the thread is making it sound like some images were made for 4:3. The OpenXcom developers used 16:10 from what I just looked at in the program.

The walls in Arena seem to be square, not tall, when looking at them in DOSBox at 640x480. Does this agree with what you were saying? I think Arena won't look perfect here either way, though, because I have a feeling that some art assets were not made for the 4:3 aspect ratio. I guess my point is that, to me, it looks like some images look worse here in 4:3 than 16:10.

I'll still end up normalizing the UI coordinates anyway because that's a more robust design. I should also redesign the blitting so that there is only one SDL_BlitScaled() function at the end, instead of lots of little SDL_BlitScaled() calls everywhere in between (this should really help CPU performance, too).

[kcat]

For the UI elements, it ultimately comes down to whether or not the engine drew the UI without software scaling (which was typical at the time because stretching was pretty costly, and the pixel resolution didn't make it look particularly good; drawing the UI images as-is was an easy save). If it didn't software scale, then we'd need to apply the vertical scaling to maintain the original look people would've had when playing it.

The 3D world view is a separate matter. Since it's drawing stretched anyway, it is possible the projection accounted for the 16:10 buffer on a 4:3 display and rendered the walls and floors slightly flattened, so when it was shown on the screen it would be stretched and look correct again. I have no idea if it did or not though, but considering Doom didn't, I would assume it wasn't standard practice for early 3D/2.5D engines.

[afritz1]

I'm brainstorming a way to do this surface scaling more efficiently.

Right now, the only surface for panel drawing is the current window's (SDL_Surface *dst), which means all surfaces need to be scaled to fit that every frame (convenient but inefficient). I'd like to have an intermediate 320x200 surface for doing non-scaled blits instead (which is faster), and then scale that surface onto the window surface at the end each frame.

But then when does the game world get drawn in the process? It should be drawn before the UI, but at the window resolution, which is after the UI is drawn. And scaling the 320x200 surface onto the game world surface would require transparency, which is far too expensive for such a big surface. Maybe each panel would be given two SDL_Surface* arguments? But that's leaning on the disorganized side.

[Ragora]

I wonder if you could use this: https://wiki.libsdl.org/SDL_RenderSetScale?highlight=%28%5CbCategoryRender%5Cb%29%7C%28CategoryEnum%29%7C%28CategoryStruct%29

It's an odd prospect, but it in theory might work if you can get your UI elements positioned correctly. Though I'm not completely sure what the affects of that would be, I'm expecting draws to be scaled by those factors until it's otherwise set to something else, where it takes affect immediately. But it might not do what I think it does. If it has problems with positioning, I wonder if you could also use this: https://wiki.libsdl.org/SDL_RenderSetViewport?highlight=%28%5CbCategoryRender%5Cb%29%7C%28CategoryEnum%29%7C%28CategoryStruct%29

I'd say to experiment with those and see if they can produce the result you're looking for. For the second thing there, I'm looking for that to be used on a per-element basis and then you reset the viewport to encompass the entire screen at the end of every frame.

Edit So far I'm not really seeing any difference when I insert the call.

Actually, I've managed to produce a noticeable difference but it looks like it might not work the way we need it to: https://i.imgur.com/zasTHPR.png

I have one last idea, though.

[afritz1]

Hmm. I wonder. A combination of those SDL functions could be useful sometime.

Maybe the texture manager could optionally return an SDL_Texture instead of an SDL_Surface for static images (like backgrounds) which take up the whole letterbox. That's where the most benefit would be from, because SDL_Texture is GPU accelerated. I don't want to limit the whole program to SDL_Textures, though, because SDL_Surface rendering is good enough for the most part when there isn't going to be any bulk drawing at all, really.

I just thought it'd be better to make it slightly less naive at some point, and that's what this brainstorm is all about!

[Ragora]

I've been getting results that are semi-towards what I was trying to do, but they're still not quite working right: https://i.imgur.com/UQSCRPD.png

Basically, if I had the result swapped around (the GUI contents being scaled versus the actual game world), it would probably actually be workable.

So far it's implemented as a really, really bad hack but this is what I've been doing thus far.

At the beginning of the GameWorldPanel's Draw:

SDL_RenderSetScale(this->getGameState()->renderer->renderer, 1, 1);

// Draw game world (OpenCL rendering). No need to clear the screen beforehand. this->getGameState()->getGameData()->getCLProgram().render(dst);

SDL_Surface* nativeSurface = this->getGameState()->renderer->getWindowSurface();

SDL_Rect test; test.x = 0; test.y = 0; test.w = 640; test.h = 480;

SDL_UpdateTexture(this->getGameState()->renderer->texture, &test, nativeSurface->pixels, nativeSurface->pitch);

SDL_RenderCopy(this->getGameState()->renderer->renderer, this->getGameState()->renderer->texture, nullptr, nullptr); SDL_RenderSetScale(this->getGameState()->renderer->renderer, 320, 200);

At the end of the GameWorldPanel's Draw:

// Draw cursor for now. It won't be drawn once the game world is developed enough. const auto &cursor = this->getGameState()->getTextureManager() .getSurface(TextureFile::fromName(TextureName::SwordCursor)); this->drawCursor(cursor, dst);

SDL_RenderSetScale(this->getGameState()->renderer->renderer, 1, 1);

Right now, aside from the wrong bits being scaled, it looks like I'm having problems with the video output being clipped.

[afritz1]

Well, dst and nativeSurface are pointing to the same thing. There's only one window surface, and that's the destination for all drawing.

I don't have any problems with that code causing the output to get clipped. The hack works, but I should really rework the Panel draw functions for normalized coordinates and allow some SDL_Textures to be used there as well. It'll take some time.

[Ragora]

I know they are. It's just problems with the way I'm trying to get this scaling to work. (By clipped, I mean that the actual output is smaller than the screen). A lot of that above is just copy-paste and rearranged a lot to see if I can get the output I'm looking for.

So far, though, it looks like it's just not happening.

[afritz1]

That's fine. I'm going to focus on normalized coordinates and SDL_Textures for now.

[afritz1]

I added the aspect ratio correction and normalized input coordinates in the commit referenced above.