RenderTexture To RenderTexture (ping pong effect) artefacts

[fred1024]

Having troubles with render textures.... A previous issue of this kind was closed without being fixed... even in cocos 3 (actually i wonder if it could be hardware/drivers related)

just trying to copy one render texture to another one each frame. frame 0: render a simple sprite to RTa (then dispose it, not used any more) frame 1: RTa to RTb frame 2: RTb to RTa frame 3: RTa to RTb frame 4: ... till the end of times

Of course the interest (in case you wonder why i would do that) is to apply a specific shader between the RT copies to do really nice effects ... well known technique and a real pitty it's messed up in cocos :(

without applying any shaders or transforms ... here are my results ..

after 10 to 20 frames:

after 50 to 60 frames

Have you tried yourself this kind of effect before closing the issue ?

[fred1024]

Hoooo I forgot ... -the central region is behaving just like it should -the initial sprite must be quite big for the artefact to be seen (almost the RT size) -the Render Textures big as well. (screen size is good to check the artefact)

the ping pong effect will work fine on small RTs

[Shulepov]

Also have similar issue with RenderTextures, but with ping-pong rendering during one frame. Render data to RTa -> render first pass to RTb -> render second path back to RTa. And also got a weird result.

Finally i decided to use third RenderTexture - RTa -> RTb -> RTc (them were quite small so it was applicable to me).

[fred1024]

Yes actually it's not that catastrophic as in my release version I apply a shader to fade down the colours in each RT copies, so the artefact is fading away and don't have much time to accumulate. And I don't use a full screen RT. So it's almost OK. It's just I wanted to know what could possibly be this artefact (it's not really serious to go away without knowing). Furthermore, I might need another effect with this technique and maybe without the color fade down. So it might be a real problem for me later on... .

[stevetranby]

Do you have (can you share) the create and visit code for your sample single image? Do you use beginWithClear? It's an interesting problem in that it seems if you clear and visit there shouldn't be any issue, as you mention.

[fred1024]

Yes the Texture is cleared before the copy. (which is a pity, I shouldn't have to do that as I fully overwrite those texels with the other RT with no blend). I tried with my own draw method ... but fell back to use the autodraw mechanism which in the end gave me a nicer code but exactly the same result. Here's some code, maybe you can spot something ... thanks for your advice

` void TestRTSprite::initialize() { const Size winSize = Director::getInstance()->getWinSize(); _rtBank = 0;

// Set up render targets (must be of same size)
// At each new frame a render target will be rendered in the other one
for (int i(0); i < 2; i++) {
    // A render target to draw into
    RenderTexture* rt;
    if(USE_POW2_TEX)
    {
        // test on pow2 size
        int rtSize = ccNextPOT( MAX(winSize.width * _texScaleFactor, winSize.height * _texScaleFactor) );
        rt = RenderTexture::create(rtSize, rtSize, Texture2D::PixelFormat::RGBA8888, 0);
    }
    else {
        // test on any window size
        rt = RenderTexture::create((int)(winSize.width * _texScaleFactor), (int)(winSize.height * _texScaleFactor), Texture2D::PixelFormat::RGBA8888, 0);
    }
    rt->retain();
    rt->setAutoDraw(true);
    rt->setKeepMatrix(true);
    rt->setAnchorPoint(Vec2::ANCHOR_BOTTOM_LEFT);
    rt->getSprite()->setAnchorPoint(Vec2::ANCHOR_BOTTOM_LEFT);
    rt->getSprite()->setScale(1.0f / _texScaleFactor);

    _renderTextures[i] = rt;
}

// compute viewport
const Size rtSize = _renderTextures[0]->getSprite()->getTexture()->getContentSizeInPixels();
float ratio = MIN(rtSize.width / winSize.width, rtSize.height / winSize.height);
float rtZoneWidth = winSize.width * ratio;
float rtZoneHeight = winSize.height * ratio;

// apply to all textures
for (auto rt : _renderTextures) {
    rt->setVirtualViewport(Vec2(0.0f, 0.0f),
                           Rect(0.0f, 0.0f, winSize.width, winSize.height),
                           Rect(0.0f, 0.0f, rtZoneWidth, rtZoneHeight));

    rt->beginWithClear(0.0f, 0.0f, 0.0f, 1.0f);
    rt->end();

    rt->getSprite()->setTextureRect(Rect(0, 0, rtZoneWidth, rtZoneHeight));
}

// The sprite that will be used as render source for RT to RT copy
_swapSprite = Sprite::createWithTexture(_renderTextures[0]->getSprite()->getTexture());
_swapSprite->retain();
_swapSprite->setBlendFunc(BlendFunc::DISABLE);
_swapSprite->setFlippedY(true);
_swapSprite->setScale(winSize.width / rtZoneWidth, winSize.height / rtZoneHeight);
_swapSprite->setAnchorPoint(Vec2::ANCHOR_BOTTOM_LEFT);

// The start up sprite (initial sprite rendered once and only in the first RT , the GHOST)
_sprite = Sprite::createWithSpriteFrameName(_spriteFrameName);
_sprite->retain();
_sprite->setAnchorPoint(Vec2::ANCHOR_MIDDLE);
_sprite->setPosition(winSize.width * 0.5, winSize.height * 0.5);
_sprite->setScale(15.0);  // must fit almost the RT to see the artefact

}

void TestRTSprite::update(float p_dt) { // no need to use bilinear between copies ! Texture2D::TexParams tNearestParams; tNearestParams.magFilter = tNearestParams.minFilter = GL_NEAREST; tNearestParams.wrapS = tNearestParams.wrapT = GL_CLAMP_TO_EDGE;

// use bilinear for result on screen !
Texture2D::TexParams tLinearParams;
tLinearParams.magFilter = tLinearParams.minFilter = GL_LINEAR;
tLinearParams.wrapS = tLinearParams.wrapT = GL_CLAMP_TO_EDGE;

// RT mechanisme
if (!_bFirstRTRendered)
{
    // first go, init render texture 0
    _renderTextures[_rtBank]->addChild(_sprite);
    addChild(_renderTextures[_rtBank]);
    _bFirstRTRendered = true;
}
else
{
    if (_sprite != nullptr)
    {
         // We can release the sprite we don't need it anymore
        _sprite->removeFromParent();
        _sprite->release();
        _sprite = nullptr;
    }
    _swapSprite->removeFromParent();

    // RTs cycling
    for(const auto &obj : getChildren())
    {
        if (obj == _renderTextures[_rtBank])
        {
            removeChild(_renderTextures[_rtBank]);
            _swapSprite->setTexture(_renderTextures[_rtBank]->getSprite()->getTexture());
            _renderTextures[_rtBank]->getSprite()->getTexture()->setTexParameters(tNearestParams);
            _renderTextures[_rtBank ^ 0x01]->addChild(_swapSprite);
            _renderTextures[_rtBank ^ 0x01]->getSprite()->getTexture()->setTexParameters(tLinearParams);
            addChild(_renderTextures[_rtBank ^ 0x01]);
            break;
        }
    }
}
_rtBank ^= 0x01;

} `

[DamienBivaud]

bump. Having the same issue here... hoping for some guidelines with rendertextures

[stevetranby]

@fred1024 You're code is missing some initialization or at least the header declaring your variables. It would help if you uploaded the source sprite sheet as well, if possible.

I'll try to look at it today regardless and see if I can create some dummy data.

[fred1024]

@stevetranby ... yes this code is part of our engine so I tried to give you the most i could before it got to big. Basically TestRTSprite is a Node just call initialize() in the constructor. The sprite comes from a TexturePaker sprite sheet. The sprite in itself was 119 x 94, so quite small (why i had to scale it by 15 to fill almost the RT).

[stevetranby]

@fred1024 got it working and behaving like your error image sample, I'm using the cocos2d-x logo so it only stretches vertically, but should be enough to try a few things. It does seem like there's a fundamental problem somewhere, so no promises I'll find any fix. Prob some filtering+precision issue like you though? Glad it's not too important for you. I prob won't spend too much time on it, but I'll dive a little deeper.

p.s. using bilinear filtering is effectively applying the "blur" effect repeatedly. It does look pretty cool though.

[stevetranby]

For reference here's one discussion with some same and also related issues. http://discuss.cocos2d-x.org/t/3-0-beta-2-rendertexture-stretching/11618/16

[stevetranby]

@fred1024 While I did change your test code to simplify it for testing I think the following worked for me. I know many people don't like changing the engine code, so hope it works for you. This very well may be a hack and probably doesn't work well if the sprite to render is a child of parent node(s) or if the scene position has changes, or you want to render out from a camera ... I'll post this on the forum so we can discuss further. Would love to know if you can test it out and see if it fixes your specific issue.

// CCRenderTexture.cpp : 568
director->loadIdentityMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_PROJECTION);
Mat4 orthoMatrix;
Mat4::createOrthographicOffCenter(0, texSize.width, 0, texSize.height, -1024, 1024, &orthoMatrix);
director->multiplyMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_PROJECTION, orthoMatrix);

My guess is that there are some precision or slight math issues related to setting up the projection for the render texture. Because it seems there are no real issues outside of this ping-pong drawing render texture into another render texture my guess would be any issue is relevant only when reading out FBO textures.

The fix makes sense to me, but I'm not yet sure why the issue was there in the first place.

[fred1024]

Thanks for your feedback ...I am a bit in a rush but it looks like you've found something has i got some improvements ... :) ... I couldn't go further today but I will try to find out the reason of this behaviour. I wouldn't have bet on a transformation problem as the artefact was noticeable inside triangles not specially at the edges and the distortion was not homogenous or I miss something...

[stevetranby]

No worries. I'll make sure to say again that this may be coincidental and entirely a hack and a change the will break other uses of RenderTexture. Maybe the solution will be to provide a way to define the projection, similar to how you can define the viewport. I'll test a few more images, try a sprite sheet frame, and vary the various sizes of the RT, viewport, and such. If I find anything that results in a incorrect display I will report back. Otherwise I'll think about submitting a PR to add support for supplying a custom projection.

[fred1024]

Yes @stevetranby , you are right .. crazy... I just added 3 lines in visit of RenderTexture and it works real fine :)

` code... in RenderTexture2D::visit()

Director::Projection projBackup = director->getProjection();
director->setProjection(Director::Projection::_2D);

_sprite->visit(renderer, _modelViewTransform, flags);
if (isVisitableByVisitingCamera())
{
    draw(renderer, _modelViewTransform, flags);
}

director->setProjection(projBackup);
director->popMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW);`

In my scene configuration (no rotations, source rt at the right Z) the rendering in RT should work with both Projection::_2D and Projection::_3D but I get no artefact in _2D and the famous artefact in _3D.

I compared the 2 projection matrix and saw nothing weird

ok (M2d) 0.000733 0.000000 0.000000 -1.000000 0.000000 0.001302 0.000000 -1.000000 0.000000 0.000000 -0.000977 0.000000 0.000000 0.000000 0.000000 1.000000

failing (M3d): 0.975589 0.000000 0.000000 -1330.215088 0.000000 1.732051 0.000000 -1330.215088 0.000000 0.000000 -1.009588 1320.667114 0.000000 0.000000 -1.000000 1328.030273

the Ws are quite different (which is alright) but those two matrix should give me the same result for my RTsprite at z = 0 and with nulled angles. if I apply those matrix to this vector Vec4(100.0f, 100.0f, 0.0f, 1.0f) I get :

M2d * V = -0.926659 -0.869792 0.000000 1.000000

M3d * V = -1232.656250 -1157.010010 1320.667114 1328.030273 M3d * V = -0.928184 -0.871223 0.994456 1.000000 (divided by W)

So xs and ys are almost the same (like expected) ... but zs look weird shouldn't they be as closed to each other as xs and ys are ?

So I thought there might be a problem with projection matrix settings so seeking for informations I came across this:

http://discuss.cocos2d-x.org/t/ccrendertexture-rendering-artifacts/3966

, where it is said "the quad is rendered either slightly scaled up or down"

Do you think there could be a problem with the way the 3d projection matrix is computed ? Having a look in Director I'v found quite weird stuff like this one (fixing issue 1334 that i couldn't find on the net):

Mat4::createPerspective(60, (GLfloat)size.width/size.height, 10, zeye+size.height/2, &matrixPerspective);

what does the height of the window is doing in the farClip calculation ? what do you think ?

Thanks for your help, I am happy to have found somebody to speak about that :)