Camera thoughts

[st-pasha]

So, I've been pondering about the camera and how to integrate it with Transform2D and had some thoughts about the general direction in which this class can be developed:

1. Camera and Viewport can be merged into a single entity. This is both from the efficiency standpoint (currently both the camera and the viewport will translate/zoom the canvas), and from the practical standpoint: currently both classes try to map between the screen coordinates and world coordinates, and also in the real-world a viewport is simply a property of a camera.

2. Camera shouldn't be a property of the Game, instead, it could be a simple Component. There are several reasons for this: first, not all games (or game views) need a camera. For example, a simple puzzle game where everything fits on a screen, or an inventory screen, or character view, or trade UI, etc. -- all don't need a camera. At the same time, there could be situations where multiple cameras are needed. For example: a split-screen multiplayer game; a game which allows magnification of a small part of a screen, such as magnifying glass functionality, or a sniper scope; minimap functionality (minimap is a camera into a separate zoomed-out map of the world); a game which shows the views from multiple security cameras at the same time; and so on. Even now, in order to distinguish components that shouldn't be affected by the camera, we introduced the .isHud flag. But that flag is unnecessary if we simply keep the "Hud" components in a separate component tree.

3. Camera can be made more modular. Currently, the camera comes with many built-in features that make it really easy to implement the most common camera behaviors: move, follow, shake. However, there is no easy way to augment or add to this functionality. For example, if I wanted the camera that follows the character horizontally but not vertically, there is no way to do that. Or if I wanted a camera that zooms out when the character gains speed (like in GTA). Or a camera that can smoothly zoom in and center on a certain object. Or a Perlin-noise-based shake. Or a world boundary that is circular instead of rectangular (someone asked for that), or an even weirder shape. One possibility here is to split the "camera behavior" logic into a separate class or several classes, and allow those to be overridden and swapped out at will. I am not sure whether the effects (such as shake) should be kept separate from the "normal" behaviors such as follow or apply-world-boundary.

[spydon]

1. I think this could be a good idea, how would it work if you'd want to implement a new viewport for example? Which is quite easy with the current structure. I don't think they would have to be merged from a efficiency stand-point, since their joined transform matrix could be applied together? But if it is made more modular like 3 suggests, then I guess this structure would be nicer anyways with less code duplication etc.

2. I think we should have Camera as a separate entity, but then we could have a CameraComponent that controls it. This way you wouldn't have to use FCS (Flame Component System) to use it, but you could use for example Oxygen (ECS). This structure we have in many places of the code base. Are you using the camera in Oxygen @wolfenrain?

3. :+1:

[wolfenrain]

1. Even in the real world a viewport can be "swapped" on a camera, so it would make more sense to have a viewport on the camera so people can still have their own custom viewport logic while still keepings the efficiency we get from matrices. But as spydon mentioned, that would also be solve-able through point 3. But keeping a viewport would make sense imho.

2. We should be wary of making everything "just a component", we have the FCS but we don't force anyone to use it. They can still use the Game class and have their own abstraction on top of it but still use many of the Flame functionality like Camera, Particles, etc.. And to answer your question @spydon OxygenGame does not come with a camera at the moment but a developer can easily add it as they would with Game(that is why I updated the docs for the Camera API).

3. A modulair system would be sweet but I wonder if there is any real benefit for it. If you want to have a custom camera it would be less "complex" for us as the engine to just allow developers to give their own camera implementation instead of supplying them with a modulair system that we have to maintain ourselves. Obviously this isn't a "reason" to not do it ;-) but if I personally wanted custom camera functionality, like the follow horizontal, I would just override certain methods and supply my own camera. So an alternate to the modular system would be to abstract our camera functionality a bit better and allow developers to just override those in their own implementation

[erickzanardo]

I am not sure about camera being a component, but I believe that it could be a field of the component, specially with the changes that @spydon is doing on the game as component refactor, that way you could have different cameras for different components and that could have a similar effect. And of course, it could still be a separate entity, for people who choose the path of not using FCS

[spydon]

I am not sure about camera being a component, but I believe that it could be a field of the component, specially with the changes that @spydon is doing on the game as component refactor, that way you could have different cameras for different components and that could have a similar effect. And of course, it could still be a separate entity, for people who choose the path of not using FCS

I don't see the downside of having it as a Component also, if we do it like we have done with the Parallax for example, so that we have both Camera and CameraComponent? Because basically all we do now is calling update manually on Camera.

[erickzanardo]

I don't see the downside of having it as a Component also, if we do it like we have done with the Parallax for example, so that we have both Camera and CameraComponent? Because basically all we do now is calling update manually on Camera.

Maybe I am just used to how we use camera as a field, but it seems weird to me to have camera as a component, like how would it be composed and added to the tree? And how would it be accessed by the game components in order to change the camera values?

If it is just an attribute on the component, I see that it would just be a matter of accessing the component parent and just accessing the camera field values. Which seems way easier.

But maybe if I can see some code example/proposal of how this API as component would be, I could change my mind.

[st-pasha]

@wolfenrain

1. ... so it would make more sense to have a viewport on the camera so people can still have their own custom viewport logic

I agree. The more I think about it, it seems clear that viewports can come in various sizes and shapes: a no-op fullscreen viewport, a rectangular viewport, a circular viewport. There are even dynamic polygon-shaped viewports such as voronoi cameras. In order to support all this variety, a viewport must be a class.

So, it would make the most sense if Viewport was a property inside the Camera class, allowing the camera to integrate tightly with it.

2. We should be wary of making everything "just a component", we have the FCS but we don't force anyone to use it.

While I understand your argument, I'm not sure how to translate it into practical terms. When I say that "camera is a component", all I'm saying is that it had the update() and render() methods. Plus the ability to be added into the game tree at any point, not necessarily at the root. Maybe we can go with @spydon's suggestion and have CameraComponent separate from the Camera, but I'm not sure at this point where to separate the two (mainly because I don't know much about Oxygen, Forge2D, Bonfire, or other engines).

3. A modular system would be sweet but I wonder if there is any real benefit for it.

Here's my thinking: currently the camera's behavior consists of several separable parts. First, there's the "follow vector" logic, which tries to update the camera's position to match the position of an external vector. Second, there's "speed limit" logic, which smoothes the camera's movement by adding a fixed speed limit. Lastly, there's the "world boundary" logic which prevents the camera from leaving a certain region. In theory, a user may want to replace any subset of these. For example, instead of following a component's position, you may want to "follow" its expected position 1 second in the future -- this way you'll have larger view of the area in front of you than the area behind. The smoothing behavior may have max acceleration in addition to max speed, and maybe some logic to make an immediate jump if the follow vector teleports far away. The world boundary can be more complex than a simple rectangle.

Now, one way to allow the user to modify these behaviors is to have each of them in a separate function, and then we allow the user to override these functions in their derived Camera class. The other possibility is to have each of those behaviors in its own small class, and allow the user to replace some or all of those classes with their own implementations. It is possible that the first option is easier; however, it may leave the Camera class polluted with variables that are not needed. For example, if I want to replace a rectangular world bound with a circular one, then the Rect worldBound property will remain in the class, even though it is no longer used, and in fact, it will only create confusion.

@erickzanardo

Maybe I am just used to how we use camera as a field, but it seems weird to me to have camera as a component, like how would it be composed and added to the tree?

The way I see it, camera's job is to render the world as it can be seen from that camera. So, in the simplified form its code could be something like this:

class Camera extends Component {
  Component world;
  Viewport? viewport;
  Transform2D transform;

  void update(double dt) { 
    // camera movement logic here ...
  }
  void render(Canvas canvas) {
    canvas.save();
    viewport?.clip(canvas);
    canvas.transform(transform.matrix);
    world.render(canvas);
    canvas.restore();
  }
}

Thus, the camera is mounted into the game as follows:

Game
├─ Camera
│  └─ World
└─ HUD

Frankly, it's not that much different than what we have right now, except that the camera's "render" logic is inside the Game class and is invoked manually. The benefit of having a Camera as a component is not very clear from this picture, but imagine that you want to have several cameras in the game. For example, the main camera shows the player character on the map, and then another camera shows the minimap in the top-right corner. Or imagine the character can cast a portal spell, and you want the portal to display a tiny view of the place where the portal leads: simply put a second camera component inside the game world and surround it with shiny sparkles. Or maybe you want to have a mirror that would reflect your character as it walks by, or there could be reflections in the water.

When having multiple cameras viewing the same world, we need to be careful not to fall into infinite recursion, and not to update the world more often than necessary.

As for accessing the camera, there could be two options: either you know exactly where it is in the game tree (e.g. gameRef.children[0]), or when creating the camera you store the reference to it before adding it to the game. The reference can be stored wherever is most convenient: either with the root Game class, or on the CharacterController, etc. And if the game doesn't use camera at all, then nothing needs to be stored.

[st-pasha]

One other thing that the camera should do (but doesn't right now) is to be able to limit rendering (and possibly updating) to only those things that will be visible on the screen. This is especially important for large open-map worlds like Minecraft.

[wolfenrain]

@st-pasha

While I understand your argument, I'm not sure how to translate it into practical terms. When I say that "camera is a component", all I'm saying is that it had the update() and render() methods. Plus the ability to be added into the game tree at any point, not necessarily at the root. Maybe we can go with @spydon's suggestion and have CameraComponent separate from the Camera, but I'm not sure at this point where to separate the two (mainly because I don't know much about Oxygen, Forge2D, Bonfire, or other engines).

Basically we have two "things" in Flame, we have the raw functionalities like Particle and Camera and then we have their FCS Component counterpart, they use the raw functionalities and wrap them in a way that people who use the Flame FCS can easily use them.

Oxygen for example is an ECS implementation where I am writing a flame bridge for. Meaning that if I want to use Particle or Camera I need to use the raw functionalities and that is only possible when they are not tied into the Flame Component System. So my main concern is that we should look out for making "everything" a FCS component.

What @spydon suggested is kinda what I mean as well, we can indeed make a Camera Component, but it should not replace Camera but just wrap the raw functionality instead.

Here's my thinking: currently the camera's behavior consists of several separable parts. First, there's the "follow vector" logic, which tries to update the camera's position to match the position of an external vector. Second, there's "speed limit" logic, which smoothes the camera's movement by adding a fixed speed limit. Lastly, there's the "world boundary" logic which prevents the camera from leaving a certain region. In theory, a user may want to replace any subset of these. For example, instead of following a component's position, you may want to "follow" its expected position 1 second in the future -- this way you'll have larger view of the area in front of you than the area behind. The smoothing behavior may have max acceleration in addition to max speed, and maybe some logic to make an immediate jump if the follow vector teleports far away. The world boundary can be more complex than a simple rectangle.

Now, one way to allow the user to modify these behaviors is to have each of them in a separate function, and then we allow the user to override these functions in their derived Camera class. The other possibility is to have each of those behaviors in its own small class, and allow the user to replace some or all of those classes with their own implementations. It is possible that the first option is easier; however, it may leave the Camera class polluted with variables that are not needed. For example, if I want to replace a rectangular world bound with a circular one, then the Rect worldBound property will remain in the class, even though it is no longer used, and in fact, it will only create confusion.

I am up for either tbh, the only concern I have with it "is it necessary". Do we need to design a modulair system or can we just use functions that are overridable. We have to keep a "maintainability" index as well apart from the "functionality" index.

It might even be viable to have both options. Flame has just a base camera that has certain methods and its render/update and a separate package could be provided for a flame_modulair_camera structure :-). That separates the maintainability concern a bit because we can incrementally fix and release things. But as I said before I am up for any as long as we add value for the end user :-)

[st-pasha]

Closing, see the final proposal in #1161.