Some 3D matrix math...

[icculus]

Passing on a tweet from https://twitter.com/albertvaka/status/1673825758941179905 for discussion...

Can we get a way to specify a camera transformation matrix in SDL3? It's quite common to pan, zoom or rotate the whole view in a game.

[slouken]

Adding 3D matrix math is a gnarly mess we shouldn't get into. There's so much functionality that someone might want, we don't want to also become a full 3D math library. The only 3D math we should add is any we need internally, e.g. for the new GPU code.

[madebr]

I think the future SDL_gpu should just make sure its data types are compatible with other libraries.

[expikr]

Would there be a place for SDL to make something like this easier/more accessible for those who don't keep 3D math in the back of their mind at all times?

https://github.com/godotengine/godot/pull/66178#issuecomment-1616756551

[albertvaka]

I requested this originally. Maybe there's a way to offer camera movement/zoom without exposing the matrix math?

[expikr]

Movement -- a bit tricky to design an API for because you don't want to force people to be locked into the cargo cult of 4x4 view matrices that had no good reason to exist ever since OpenGL moved away from fixed function pipelines 20 years ago.

Zoom -- is achieved by right-multiplying the projection matrix with a zw-squeeze like so:

$$ P{rojMagnified} = P{roj} \cdot \begin{pmatrix} 1 & & & \ & 1 & & \ & & \frac{1}{M} & \ & & & \frac{1}{M} \ \end{pmatrix} $$

Or you can think of it as compressing the cameraspace z-component (and adjusting the clipping metric accordingly):

$$ m_{agnifiedVertex} = \begin{pmatrix} X \ Y \ \frac{Z}{M} \ \frac{1}{M} \end{pmatrix} = \begin{pmatrix} 1 & & & \ & 1 & & \ & & \frac{1}{M} & \ & & & \frac{1}{M} \ \end{pmatrix} \begin{pmatrix} X \ Y \ Z \ 1 \end{pmatrix} $$

[albertvaka]

Assuming there's no way to rotate the camera, movement should be easy to add too: it's just an X and Y displacement. This can be on the API without exposing any matrix math.

If we want to allow both movement and rotation of the camera, then I think there's no way to do it without exposing matrices on the API (because the order in which you apply the transformations gives different results), which the authors seem to be against.

So, maybe this issue could be renamed to "Add camera zoom and movement API" so it doesn't involve matrices @slouken?

[expikr]

Assuming there's no way to rotate the camera, movement should be easy to add too: it's just an X and Y displacement. This can be on the API without exposing any matrix math.

If we want to allow both movement and rotation of the camera, then I think there's no way to do it without exposing matrices on the API (because the order in which you apply the transformations gives different results), which the authors seem to be against.

So, maybe this issue could be renamed to "Add camera zoom and movement API" so it doesn't involve matrices @slouken?

Interfacing camera movement is the same degree of "exposing the math" as interfacing camera rotation. The problem is that people may want to organize it differently from the cargo-cult convention which pre-multiplies the "just an x and y displacement" with the camera rotation inverse. Too many people grew up copying this incredibly inefficient and janky way of organizing their matrices, so they would want the API to be designed to work like that, whereas other people who want their game to work correctly no matter how far out you moved in the world would hate being locked into this way and having to add their own workaround to undo the defaults.

[albertvaka]

Then it seems unavoidable to add a function that takes a matrix if we want to have the concept of a "camera". I think it's still worth doing because a camera is a very useful thing to have, but up to you folks to decide.

Edit: alternatively, a super simplified version of this would be an API that allows moving the camera but not rotating nor zooming it, in which case it would be possible to avoid the matrix multiplication order problem that you mention (at the expense of a less versatile API, but still more versatile than not having a camera IMO).

[expikr]

In the "traditional way", if your camera is already rotated in any way, any movement will always need to be multiplied by the existing rotation, so in any case you will always need access to the "full matrix". The main issue here is not about arbitrarily gimping some functionality to make it "easier to impllement" (it does not), but rather that the concept of a camera is at a level above what SDL serves, so implementation of the concept will be an arbitrary "lock-in" for those who wishes to implement cameras in a different manner.

It makes more sense to provide math helper functions rather than some built-in interface. But at that point people might just import different libraries of their choice for those helper functions (or just write their own), since 3D math isn't really that complicated, you can wrap your head around its entirety in an afternoon. The tricky part is dealing with the janky cargo-cult conventions of interfacing with fixed-function parts of the graphics API

[expikr]

@albertvaka what could be helpful is if you can describe common scenarios or pain points where you felt "I just wish there's a quick plug-and-play way to quickly try things out without all the rituals", it might not be appropriate to have them directly as part of SDL's functionality but it could be useful to brainstorm what the underlying use cases would be and come up with indirect solutions.

[albertvaka]

Here goes an example:

Let's say I'm drawing a scene made of sprites. For each sprite I have an X and Y position. But my scene is bigger than my screen, so I provide a way to move through the scene. As I move my view in one direction, some sprites enter into view as others leave the screen from the other side.

Imagine a side-scrolling game, where my character moves from left to right and my view is always centered on the character. Or a top-down strategy game where I can "scroll" in both axis through the map.

I know the X and Y position for each sprite in scene coordinates. However, to draw the sprite I now need to convert those X and Y coordinates to screen coordinates. If I'm just applying a translation, this just means subtracting the view's X and Y coordinates to each sprite X and Y coordinates before drawing it. But if I also want to zoom-i or rotate the screen then I need to do some maths for each sprite to calculate their position on screen.

I think this use case is very common. So if SDL did this coordinate transformation for me, not only would simplify the code in many applications but it would also enable running this on the GPU instead of the CPU.

When/if I want to draw things in screen coordinates (eg: a UI on top of my scene), I should be able to reset the view back to 0 before doing so.

[slouken]

We're not currently planning on adding this. @icculus can weigh in if it makes sense with the SDL GPU API though!

[icculus]

if it makes sense with the SDL GPU API though!

My current thought is no; it could certainly be useful to someone using a 3D API, but it's not useful to the API itself, and it's easy ("easy") to get matrix math code into an app from somewhere else.

However, I do think this could be useful as a single-header library. I haven't looked, but I would be surprised if such as thing didn't already exist outside of SDL. I don't have time to sleep recently, though, let alone take this project on.