implementing mirror in SupertuxKart

[luffah]

Description

I tried to implement a mirror to check backwards in STK.

I succeeded to add sub cameras to a camera (other views in located viewports) but irrlicht doesn't allow to flip rendering on vertical axis (mirror effect).

The only ergonomic option (that is not satisfying) is to put a double view, one showing what is on the left side at left, and the other on rigth side at right. (Note: only 2 cameras are active in this picture — the main one and the one for the mirror, the split comes from a kind of bug from irrlicht)

Is there some future project to use godot or urho ?

Version STK v1.1

[luffah]

related to https://github.com/supertuxkart/stk-code/issues/4134, https://github.com/supertuxkart/stk-code/issues/3246, https://github.com/supertuxkart/stk-code/issues/4238

[Benau]

No intention for godot

[luffah]

So the only way to add a mirror is to mimic kartrider crazy racing. A camera (that is shown when and) that shows backwards when an object of interest is get going closer (kart, ball or flag).

[luffah]

For curious peoples (@risostk ). Here a patch to mimic kartrider crazy racing dynamic mirror (prototype - does not show the ball or the flag).

diff --git a/src/graphics/camera.cpp b/src/graphics/camera.cpp
index c11a373..740795a 100644
--- a/src/graphics/camera.cpp
+++ b/src/graphics/camera.cpp
@@ -81,6 +81,9 @@ Camera* Camera::createCamera(unsigned int index, CameraType type,
     {
     case CM_TYPE_NORMAL: camera = new CameraNormal(CM_TYPE_NORMAL, index, kart);
                                                                  break;
+    case CM_TYPE_MIRROR:
+                         camera = new CameraNormal(CM_TYPE_MIRROR, index, kart);
+                                                                 break;
     case CM_TYPE_DEBUG:  camera = new CameraDebug (index, kart); break;
     case CM_TYPE_FPS:    camera = new CameraFPS   (index, kart); break;
     case CM_TYPE_END:    camera = new CameraEnd   (index, kart); break;
@@ -169,6 +172,20 @@ void Camera::setKart(AbstractKart *new_kart)
 void Camera::setupCamera()
 {
     m_viewport = irr_driver->getSplitscreenWindow(m_index);
+    if (m_type == CM_TYPE_MIRROR) {
+        float x = m_viewport.UpperLeftCorner.X;
+        float y = m_viewport.UpperLeftCorner.Y;
+        float w = m_viewport.getWidth();
+        float h = m_viewport.getHeight();
+        // left & right split top
+        // m_viewport = core::recti( x+(w*0.25), y, x+(w*0.75), y+(h*0.25));
+
+        // left & right split bottom
+        // m_viewport = core::recti( x+(w*0.25), y+(h*0.75), x+(w*0.75), y+h);
+
+        // bottom
+        m_viewport = core::recti(x, y+(h*0.6), x+(w*0.4), y+h);
+    }
     m_aspect = (float)((float)(m_viewport.getWidth()) / (float)(m_viewport.getHeight()));

     m_scaling = core::vector2df(
@@ -190,6 +207,10 @@ void Camera::setupCamera()
  */
 void Camera::setMode(Mode mode)
 {
+    if (m_type == CM_TYPE_MIRROR) {
+        m_mode = CM_REVERSE;
+        return;
+    }
     if (mode == m_mode) return;
     // If we switch from reverse view, move the camera immediately to the
     // correct position.
@@ -214,6 +235,9 @@ void Camera::setMode(Mode mode)
  */
 Camera::Mode Camera::getMode()
 {
+    if (m_type == CM_TYPE_MIRROR) {
+        m_mode = CM_REVERSE;
+    }
     return m_mode;
 }   // getMode

diff --git a/src/graphics/camera.hpp b/src/graphics/camera.hpp
index 3be0825..156d98e 100644
--- a/src/graphics/camera.hpp
+++ b/src/graphics/camera.hpp
@@ -52,6 +52,7 @@ public:
     enum CameraType
     {
         CM_TYPE_NORMAL,
+        CM_TYPE_MIRROR,
         CM_TYPE_DEBUG,         //!< A debug camera.
         CM_TYPE_FPS,           //!< FPS Camera
         CM_TYPE_END            //!< End camera
@@ -127,6 +128,9 @@ protected:
     */
     AbstractKart   *m_kart;

+    std::vector<Camera*> m_children_cameras;
+    std::vector<bool> m_children_cameras_toggle;
+
     static Camera* createCamera(unsigned int index, CameraType type,
                                 AbstractKart* kart);

@@ -221,6 +225,27 @@ public:
     /** Returns the scaling in x/y direction for this camera. */
     const core::vector2df& getScaling() const {return m_scaling; }

+    void setChildrenCam(bool val) {
+      for (int i=0; i<(int)m_children_cameras.size(); i++) {
+        m_children_cameras_toggle[i] = val;
+      }
+    }
+    bool isChildrenToggle(int i) { return m_children_cameras_toggle[i]; }
+    std::vector<Camera*> getChildrenCameras() { return m_children_cameras; }
+    std::vector<Camera*> getActiveChildrenCameras() {
+      std::vector<Camera*> result;
+      for (int i=0; i<(int)m_children_cameras.size(); i++) {
+        if (m_children_cameras_toggle[i]) {
+          result.push_back(m_children_cameras[i]);
+        }
+      }
+      return result;
+    }
+    void addChild(Camera* scam) {
+      m_children_cameras.push_back(scam); 
+      m_children_cameras_toggle.push_back(false);
+    }
+
     // ------------------------------------------------------------------------
     /** Returns the camera scene node. */
     scene::ICameraSceneNode *getCameraSceneNode() { return m_camera; }
diff --git a/src/graphics/camera_normal.cpp b/src/graphics/camera_normal.cpp
index 6770bb6..d8525c4 100644
--- a/src/graphics/camera_normal.cpp
+++ b/src/graphics/camera_normal.cpp
@@ -68,6 +68,10 @@ CameraNormal::CameraNormal(Camera::CameraType type,  int camera_index,
         btTransform btt = kart->getSmoothedTrans();
         m_kart_position = btt.getOrigin();
         m_kart_rotation = btt.getRotation();
+        if (type == CM_TYPE_NORMAL) {
+          Camera *camera_mirror = new CameraNormal(CM_TYPE_MIRROR, 0, kart);
+          addChild(camera_mirror);
+        }
     }
 }   // Camera

@@ -316,6 +320,7 @@ void CameraNormal::update(float dt)
         core::vector3df current_target = (m_kart->getSmoothedXYZ().toIrrVector()
                                        +  core::vector3df(0, above_kart, 0));
         m_camera->setTarget(current_target);
+        setChildrenCam(false);
     }
     else // no kart animation
     {
@@ -323,6 +328,35 @@ void CameraNormal::update(float dt)
         bool  smoothing;
         getCameraSettings(&above_kart, &cam_angle, &side_way, &distance, &smoothing, &cam_roll_angle);
         positionCamera(dt, above_kart, cam_angle, side_way, distance, smoothing, cam_roll_angle);
+
+        World* world = World::getWorld();
+        World::KartList karts = world->getKarts();
+
+        bool set_child_cam=false;
+        if (karts.size()>1){
+            Vec3 kart_pos=m_kart->getXYZ();
+            // Find the direction a kart is moving in
+            btTransform trans = m_kart->getTrans();
+            Vec3 direction(trans.getBasis().getColumn(2));
+            Vec3 kart_posb=kart_pos+direction;
+            for (unsigned int i = 0; i < karts.size(); i++)
+            {
+                const AbstractKart *kart = karts[i].get();
+                if (kart == m_kart) continue;
+                Vec3 tmp_pos = kart->getXYZ();
+                Vec3 to_tmp = tmp_pos-kart_pos;
+                Vec3 to_tmpb = tmp_pos-kart_posb;
+                if ((to_tmpb.length() > to_tmp.length()) && (to_tmp.length()<50) && (to_tmp.dot(to_tmpb) > to_tmpb.length())) {
+                  set_child_cam=true;
+                  break;
+                }
+            }
+        }
+        setChildrenCam(set_child_cam);
+    }
+    for (auto const&  cam: getActiveChildrenCameras())
+    {
+      cam->update(dt);
     }
 }   // update

diff --git a/src/graphics/shader_based_renderer.cpp b/src/graphics/shader_based_renderer.cpp
index 095da29..c334fcf 100644
--- a/src/graphics/shader_based_renderer.cpp
+++ b/src/graphics/shader_based_renderer.cpp
@@ -769,6 +769,34 @@ void ShaderBasedRenderer::render(float dt, bool is_loading)
         // Save projection-view matrix for the next frame
         camera->setPreviousPVMatrix(irr_driver->getProjViewMatrix());

+        for (auto const& scam: camera->getActiveChildrenCameras())
+          {
+            scene::ICameraSceneNode * const scamnode = scam->getCameraSceneNode();
+
+            scam->activate(!CVS->isDeferredEnabled());
+            rg->preRenderCallback(scam);   // adjusts start referee
+            irr_driver->getSceneManager()->setActiveCamera(scamnode);
+
+            computeMatrixesAndCameras(scamnode, m_rtts->getWidth(), m_rtts->getHeight());
+            if (CVS->isDeferredEnabled())
+            {
+              renderSceneDeferred(scamnode, dt, track->hasShadows(), false); 
+            }
+            else
+            {
+              renderScene(scamnode, dt, track->hasShadows(), false); 
+            }
+
+            if (CVS->isDeferredEnabled() && !is_loading)
+            {
+              renderPostProcessing(scam, false);
+            }
+
+            // Save projection-view matrix for the next frame
+            scam->setPreviousPVMatrix(irr_driver->getProjViewMatrix());
+          }
+
+
         PROFILER_POP_CPU_MARKER();

     }  // for i<world->getNumKarts()

[Benau]

If you are intented to be merged, please make the mirror only show the backwards if activated, if you make it auto follow some object then it's cheating

[Benau]

also render a scene twice will need to be evaluated very carefully to make it doesn't break anything (next frame it draw in full), so if you make it too complicated...

[Benau]

or better: Allow the look back button to render to miniscreen instead (configurable option between full screen (current ony) / miniscreen)

[luffah]

If you are intented to be merged, please make the mirror only show the backwards if activated, if you make it auto follow some object then it's cheating

I only experiment, but the miniscreen seems usefull only if there is a part of cheating :p it just say, "hey look !" "listen" "there is something backwards.

Rendering the scene twice begin to be slow in heavy maps like "dumas soccer".

If we got clear spec maybe i ll try to got that merged. The code is here, for other motivated peoples.

[Benau]

actually samuncle's "raytrace reflexion" will be broken for double rendering

[risostk]

This is very cool! Thanks. I tried it on Oliver's Math Class, the detection of karts behind does not always work well. I think the reason is that it is a small map. How about this?

if (karts.size()>1)
{
    // Find the direction a kart is moving in
    btTransform trans = m_kart->getTrans();
    Vec3 direction(trans.getBasis().getColumn(2));
    for (unsigned int i = 0; i < karts.size(); i++)
    {
        const AbstractKart *kart = karts[i].get();
        if (kart == m_kart) continue;
        // other kart's position in kart's location coordinates
        Vec3 tmp_pos = trans.inverse()(kart->getXYZ());
        // other kart's direction
        Vec3 tmp_dir = kart->getTrans().getBasis().getColumn(2);
        if (tmp_pos.getX()>-10 && tmp_pos.getX()<10 &&
            tmp_pos.getZ()>-50 && tmp_pos.getZ()< 0 && // behind kart
            direction.dot(tmp_dir)>0) // same direction
        {
            set_child_cam=true;
            break;
        }
    }
}

Showing the mirror when a basketball is coming is useful, I think. Always showing the mirror, as Benau suggested, is also good. Just curious, would it be possible to use a different projection, e.g. zoom in and cut the left and right sides, for the mirror? I think most of the time the most interesting thing is right behind, no need a very wide view.

[luffah]

Just curious, would it be possible to use a different projection, e.g. zoom in and cut the left and right sides, for the mirror?

There is 2 ways:

In camera.cpp , changing focal distance:

 void Camera::setupCamera()
{
    m_viewport = irr_driver->getSplitscreenWindow(m_index);
    float fov_factor=1.0f;
    if (m_type == CM_TYPE_MIRROR) {
        float x = m_viewport.UpperLeftCorner.X;
        float y = m_viewport.UpperLeftCorner.Y;
        float w = m_viewport.getWidth();
        float h = m_viewport.getHeight();
        // left & right split top
        // m_viewport = core::recti( x+(w*0.25), y, x+(w*0.75), y+(h*0.25));

        // left & right split bottom
        // m_viewport = core::recti( x+(w*0.25), y+(h*0.75), x+(w*0.75), y+h);

        // bottom
        m_viewport = core::recti(x, y+(h*0.7), x+(w*0.4), y+h);
        fov_factor=0.5;
    }
    m_aspect = (float)((float)(m_viewport.getWidth()) / (float)(m_viewport.getHeight()));

    m_scaling = core::vector2df(
        float(irr_driver->getActualScreenSize().Width) / m_viewport.getWidth() , 
        float(irr_driver->getActualScreenSize().Height) / m_viewport.getHeight());

    m_fov = fov_factor * DEGREE_TO_RAD * stk_config->m_camera_fov
        [RaceManager::get()->getNumLocalPlayers() > 0 ?
        RaceManager::get()->getNumLocalPlayers() - 1 : 0];

    m_camera->setFOV(m_fov);
    m_camera->setAspectRatio(m_aspect);
    m_camera->setFarValue(Track::getCurrentTrack()->getCameraFar());
}   // setupCamera

AND/OR by changing camera position. In camera_normal.cpp:227

    case CM_REVERSE: // Same as CM_NORMAL except it looks backwards
        {
            *above_kart = 0.75f;
            *cam_angle  = kp->getCameraBackwardUpAngle() * DEGREE_TO_RAD;
            *sideway    = 0;
            *distance   = (getType()==CM_TYPE_MIRROR ? 1.0f : 2.0f)*m_distance;
...

But i prefer to keep a correct focal distance to a have look on sides.

[luffah]

I agree to disable the automatic appearance/disappareance of mirror (less awesome but still useful). Still to resolve the perfect position and performances issues.

I don't know well irrlicht but maybe it is possible to replace the second camera rendering by an additional planar object that reflect all.

[luffah]

With the current patch, there is a bug in spectator mode : if you select an other player, then only the mirror part is changed (and then you observe 2 players).

[risostk]

Hi luffah, Seems there is an overlay between the scene in the mirror and the main one. Would it possible to make the mirror on top of everything else? Thanks.

[luffah]

hmmm, maybe drawing it after/during race gui drawing. As i understood, the last drawn thing is on top of everything.

All i known on irrlicht was found on http://irrlicht.sourceforge.net/forum

[risostk]

Looks the mirror is drawn after the main scene. Maybe an Irrlicht setting issue. Thanks for the link.

[luffah]

(I close this issue. Really it is not a convincing feature —could be interesting in a driving simulator—. Keep your time for interesting things.)