Unreal Bloom and Renderer Transparency issue

[fire67]

Description of the problem

I am currently using the Unreal Bloom post with a transparent renderer.

renderer = new THREE.WebGLRenderer({ alpha: true });
renderer.setClearColor(0xFF0000, 0);

Unfortunately, the Unreal Bloom pass doesn't seem to take in account the alpha channel of the coming texture when doing the blur and it apparently comes from getSeperableBlurMaterial in js\postprocessing\UnrealBloomPass.js.

The fragment's alpha channel is always set to 1.0 which results in a complete removal of the previous alpha values when doing the additive blending at the end.

This is happening in the latest version of Three.js r92 and you can experiment about this issue using the Unreal Bloom example

[WestLangley]

In getSeperableBlurMaterial()you can try a work-around:

gl_FragColor = vec4( diffuseSum / weightSum, 0.1 );\n\

Remember, it is an example, so you are free to modify it.

[fire67]

Thanks for the answer @WestLangley ! Yeah this would help keeping the transparency but would destroy the bloom effect (which is really cool btw) around the object. I think a more cleaner solution would be to apply weight on alpha channel too.

Any solution would help me a lot :)

[WestLangley]

/ping @spidersharma03

[fire67]

I achieved to find a solution for this but I would like to know your opinion @spidersharma03 and @WestLangley.

void main()
{
    vec2 invSize = 1.0 / texSize;
    float fSigma = float(SIGMA);
    float weightSum = gaussianPdf(0.0, fSigma);
    float alphaSum = 0.0;
    vec3 diffuseSum = texture2D(colorTexture, vUv).rgb * weightSum;
    for( int i = 1; i < KERNEL_RADIUS; i ++ )
    {
        float x = float(i);
        float weight = gaussianPdf(x, fSigma);
        vec2 uvOffset = direction * invSize * x;

        vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);
        float weightAlpha = sample1.a * weight;
        diffuseSum += sample1.rgb * weightAlpha;
        alphaSum += weightAlpha;
        weightSum += weight;

        vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);
        weightAlpha = sample2.a * weight;
        diffuseSum += sample2.rgb * weightAlpha;
        alphaSum += weightAlpha;
        weightSum += weight;

    }
    alphaSum /= weightSum;
    diffuseSum /= alphaSum; // Should apply discard here if alphaSum is 0
    gl_FragColor = vec4(diffuseSum.rgb, alphaSum);
}

[WestLangley]

Sorry, I have no opinion on this.

[spidersharma03]

Sorry @fire67 , was away for a few days. Yeah, what you have done seems correct. I didn't handle the transparency.

[fire67]

No problem, thank you for the answer @spidersharma03 ! While testing the effect I noticed a visual difference when pushing some values very high.

Do you have any idea where does this comes from ?

[spidersharma03]

@fire67 , Not sure actually. Do you have the changed code in some branch? I will try to take a look. Thanks!

[fire67]

@spidersharma03 No, unfortunately the code is not available on a branch. Do you want me to do it ? Just to let you know, there's nothing more than the main function in the previous post.

[jcyuan]

@fire67 @spidersharma03 hi, I tried to use your main() code to replace the old one in example but all bloom effect gone although the background takes effect it's transparent.

this.renderer = new THREE.WebGLRenderer({
        canvas: view,
        alpha:true
});

//and pass
this.$unrealBloom = new UnrealBloomPass(new THREE.Vector2(this.app.stageWidth, this.app.stageHeight), 3.2, .6, 0.7 );
this.$composer.addPass(this.$unrealBloom);

this.$blurPass = new ShaderPass(HorizontalBlurShader);
this.$blurPass.uniforms['h'].value = 0;
this.$composer.addPass(this.$blurPass);

this.$blurPass.renderToScreen = true;

please help, thanks.

[jcyuan]

ok so the problem is caused by my setting: this.$renderer.context.enable(this.$renderer.context.SAMPLE_ALPHA_TO_COVERAGE); the alpha channel of user RTT will not be handled by lowlevel for A2C in WebGL1.0 (it does in WebGl2.0)

the changed shader works well actually, thank you very much @fire67

[shokunin000]

Is it possible to increase bloom based on emissive values? Trying to utilize those values to increase strength on glowing objects but this shader code is bit above my head! Any help would be much appreciated

[robhawkes]

@fire67 Did you manage to get this working with the bloom keeping the colour of the object? As soon as I try to use the changed shader the bloom just goes pure white, rather than taking the object colour as before.

I also tried disabling rendering.alpha but the bloom was still pure white.

Any ideas?

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float weight = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    float weightAlpha = sample1.a * weight;\
    diffuseSum += sample1.rgb * weightAlpha;\
    alphaSum += weightAlpha;\
    weightSum += weight;\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    weightAlpha = sample2.a * weight;\
    diffuseSum += sample2.rgb * weightAlpha;\
    alphaSum += weightAlpha;\
    weightSum += weight;\
  }\
  alphaSum /= weightSum;\
  diffuseSum /= alphaSum;\
  gl_FragColor = vec4(diffuseSum.rgb, alphaSum);\n\
}

Before:

After:

[robhawkes]

I've been mulling this over for the past few days and decided to attempt a slightly different approach that takes the original (non-transparent) colour calculations, leaves them unmodified and then adds on top the new alpha calculations from @fire67 with a couple of minor tweaks to utilise the same weights as the colour calculations. I don't know if this is the right way to do it, and the calculations could probably be tidied up, but it works.

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float w = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    diffuseSum += (sample1.rgb + sample2.rgb) * w;\
    weightSum += 2.0 * w;\
    float weightAlpha = sample1.a * w;\
    alphaSum += weightAlpha;\
    weightAlpha = sample2.a * w;\
    alphaSum += weightAlpha;\
  }\
  gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);\n\
}

Previous approach (posted earlier in this issue):

New approach combining original logic with updated logic posted earlier in the issue:

[robhawkes]

Tidy things up a bit and it seems to work just as well:

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float w = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    diffuseSum += (sample1.rgb + sample2.rgb) * w;\
    alphaSum += (sample1.a + sample2.a) * w;\
    weightSum += 2.0 * w;\
  }\
  gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);\n\
}

I've noticed some little (single pixel) artefacts that appear once in a while but seem to be random or based on something that only occurs on some frames as they disappear on the next frame. Not sure if it's caused by the changes I made or something unrelated to this issue.

[robhawkes]

I've done some more testing and can confirm that the artefacts that I was seeing are unrelated to Three.js and the changes that I made to the bloom shader, so that's good.

[mrdoob]

/ping @spidersharma03

[spidersharma03]

I will see the results with a colored scene and come back.

[spidersharma03]

@robhawkes , I see that you initialize the alphaSum to 0. Did you simply try making something like this::

vec4 diffuseSum = texture2D( colorTexture, vUv)* weightSum;\ ... ... diffuseSum /= weightSum;

[spidersharma03]

@robhawkes , basically something like this::

                                    float fSigma = float(SIGMA);
                float weightSum = gaussianPdf(0.0, fSigma);
                vec4 diffuseSum = texture2D( colorTexture, vUv) * weightSum;
                for( int i = 1; i < KERNEL_RADIUS; i ++ ) {
                    float x = float(i);
                    float w = gaussianPdf(x, fSigma);
                    vec2 uvOffset = direction * invSize * x;
                    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);
                    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);
                    diffuseSum += (sample1 + sample2) * w;
                    weightSum += 2.0 * w;
                }
                gl_FragColor = vec4(diffuseSum/weightSum);

I need to test it though

[simongloor]

I tested the last two adjustments.

The version of @robhawkes turns grayscale in very bright areas.

The version of @spidersharma03 comes pretty close to the opaque version when the background is black, but it fades the color twice and doesn't work on bright backgrounds:

I think the problem is that the actual geometry should replace the background of the canvas while the glow should actually be added additive on top of both.

[revyTH]

Tidy things up a bit and it seems to work just as well:

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float w = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    diffuseSum += (sample1.rgb + sample2.rgb) * w;\
    alphaSum += (sample1.a + sample2.a) * w;\
    weightSum += 2.0 * w;\
  }\
  gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);\n\
}

I've noticed some little (single pixel) artefacts that appear once in a while but seem to be random or based on something that only occurs on some frames as they disappear on the next frame. Not sure if it's caused by the changes I made or something unrelated to this issue.

Hi guys, after changes from 101 to 102 it seems that the change on the fragment shader does not work anymore. Probably related to the change "Removed renderTarget and forceClear parameters from WebGLRenderer.render(). Please use .setRenderTarget() and .clear() instead before you perform the rendering. Be aware that it's now necessary to execute renderer.setRenderTarget( null ) in order to unset an active render target."

Any thought?

[robhawkes]

I'm slowly upgrading all my old experiments to work with r108 and after the comment by @revyTH I was worried my workaround for transparency in the bloom shader wasn't going to work any more. I'm unsure if something was actually broken with this in r102 but when I just added the same modified fragment shader I posted previously everything seems to work OK in r108.

For the avoidance of doubt, the background in the Three.js renderer in the screenshots is transparent, and overlaid over a Mapbox GL JS map. Without the modified fragment shader the bloom shader is opaque and you cannot see the map behind.

[robhawkes]

And for comparison, this is the same scene (slightly rotated) with the alternative approach suggest by @spidersharma03

[anton-ax]

@robhawkes Can you share this working example with mapbox map?

[fuzhenn]

I found a simpler solution:

It doesn't need to take alpha into account during gaussian blur. At the final step, when combining bloom texture with the base texture, try to caculate an approximate alpha value for bloom rgb.

void main() {
    vec4 baseColor = getTexture( baseTexture );
    vec3 bloom = getTexture( bloomTexture ).rgb;
    //approximate alpha for bloom pixel when baseColor.a is 0 (transparent)
    //you can adjust it to make it more intense or softer
    float bloomAlpha = sqrt((bloom.r + bloom.g + bloom.b) / 3.0);
    //only use bloomAlpha when baseColor.a is 0
    float alpha = mix(bloomAlpha, baseColor.a, sign(baseColor.a));
    gl_FragColor = vec4(baseColor.rgb + bloom, alpha);

    // old code:
    // gl_FragColor = ( getTexture( baseTexture ) + vec4( 1.0 ) * getTexture( bloomTexture ) );
}

rendering result seems to be promising:

[valentincognito]

@fuzhenn That's the only modification you brought to the shader and you were able to set the renderer to alpha true and use a CSS background ?

[MagicBYang]

I'm slowly upgrading all my old experiments to work with r108 and after the comment by @revyTH I was worried my workaround for transparency in the bloom shader wasn't going to work any more. I'm unsure if something was actually broken with this in r102 but when I just added the same modified fragment shader I posted previously everything seems to work OK in r108.

For the avoidance of doubt, the background in the Three.js renderer in the screenshots is transparent, and overlaid over a Mapbox GL JS map. Without the modified fragment shader the bloom shader is opaque and you cannot see the map behind.

Thanks a lot for the mapbox example. I try the same thing for a few days but always got a white untransparent scene above the mapbox map when I add your shader code. So I guess the problem is the version of three.js. I tried r116dev and r108.r108 could not work at all with a total white scene.r116dev display like below:

I stop render the unrealbloom pass the map show up again:

I cant find whats going on here if you still work in this field please show any clue to me if possible. I saw a few similar issue in mapboxgl.js project too.

[luca-viggiani]

Hi, I see that this issue is still around since about three years and is now postponed to rXXX... Is there any workaround to use unrelabloom and keep canvas transparent background applicable to r127?

[bamboofx]

i got same problem to day. What things i did and its work for me:

edit UnrealBloomPass.js

gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);

set renderer clear color:

renderer.setClearColor(0x000000, 0.0);

add clearPass

let clearPass = new ClearPass(0x000000, 0)

set renderPass clear to false

renderPass.clear = false

Note: add pass correct order

this.composer.addPass(clearPass);
this.composer.addPass(renderPass);
this.composer.addPass(unrealBloom);

THREE r128

[luca-viggiani]

i got same problem to day. What things i did and its work for me:

edit UnrealBloomPass.js

gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);

set renderer clear color:

renderer.setClearColor(0x000000, 0.0);

add clearPass

let clearPass = new ClearPass(0x000000, 0)

set renderPass clear to false

renderPass.clear = false

Note: add pass correct order

this.composer.addPass(clearPass);
this.composer.addPass(renderPass);
this.composer.addPass(unrealBloom);

THREE r128

Hi, thanks for your suggestion, however the shader won't compile:

gl.getShaderInfoLog() fragment
ERROR: 0:196: 'alphaSum' : undeclared identifier

I guess there's something else to change in UnrealBloomPass.js?

or maybe you refer to this? https://github.com/mrdoob/three.js/issues/14104#issuecomment-429664412

EDIT I managed to get it to work with this: https://github.com/mrdoob/three.js/issues/14104#issuecomment-429664412

and renderer.setClearColor(0x000000, 0.0);

no need to add dedicated ClearPass

[mbalex99]

I've created a full example. Thanks to help from: https://github.com/mrdoob/three.js/issues/14104#issuecomment-429664412

Disclaimer it's in typescript:

• Example
• Source Code

[jan9won]

I've created a full example. Thanks to help from: #14104 (comment)

Disclaimer it's in typescript:

• Example
• Source Code

Hi, Your fix worked great for me before, but when I added material.clippingPlanes to clip my objects, somehow clipping directions are being messed up. ( flipped, sometimes clipped object is half opaqe )

flipped problem for example, following code worked well alone,

material.clippingPlanes = [
    new THREE.Plane( new THREE.Vector3( 1, 0, 0 ), 0 )
]

but with bloom it becomes like this.

material.clippingPlanes = [
    new THREE.Plane( new THREE.Vector3( -1, 0, 0 ), 0 )
]

[transitive-bullshit]

Confirming @mbalex99's solution works for me as well 💪

Would it make sense to port this change back into the official example?

[mrdoob]

Would it make sense to port this change back into the official example?

Feel free to give it a try 👍

[octaharon]

For anyone who's still struggling with this, I was able to achieve a similar (though not quite the same) result with original UnrealBloomPass by placing a ShaderPass on top of it with a makeshift additive transparency shader

new ShaderMaterial({
        vertexShader: ColorifyShader.vertexShader,
        fragmentShader: "
        uniform float transparency;
        uniform sampler2D tDiffuse;

        varying vec2 vUv;

        void main() {

            vec4 texel = texture2D( tDiffuse, vUv );

            vec3 luma = vec3(0.33, 0.33, 0.33);
            float v = pow(dot( texel.rgb, luma ), 0.05); // the exponent can be parametrized to adjust brightness
            gl_FragColor = vec4( texel.rgb, texel.a*v*transparency );
        }
        ",
        transparent: true,
        blending: AdditiveBlending,
        uniforms: {
            transparency: { value: 1}, // can be parametrized as well
        },
})

this probably will be undesireable though, if the UnrealBloomPass is not the final shader in the chain

[netgfx]

I've created a full example. Thanks to help from: #14104 (comment)

Disclaimer it's in typescript:

• Example
• Source Code

Any idea if this exists for r3f or is there another way to achieve transparent background and bloom effect in r3f nowadays?

[manmohanbishnoi]

I've created a full example. Thanks to help from: #14104 (comment)

Disclaimer it's in typescript:

• Example
• Source Code

Solution provided by @mbalex99 worked for me with r136

[DOSputin]

Typescript 🤢 Reverted @mbalex99's implementation back to pure JS and it does work fine with alpha, also r136

[mrdoob]

Reverted @mbalex99's implementation back to pure JS and it does work fine with alpha, also r136

Lovely! Could you share the JS version? 🤓

[DOSputin]

Reverted @mbalex99's implementation back to pure JS and it does work fine with alpha, also r136

Lovely! Could you share the JS version? 🤓

I'm sort of new here in the Three.js scene. How would you suggest might be the best way to share? As in, what's the preference?

[DOSputin]

@mrdoob; I uploaded my reimplementation (JS version) in this gist: UnrealBloomPass.js. Should these changes be integrated into the original UnrealBloomPass and do a #14104 branch PR?

[mrdoob]

A new PR would be great actually. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request

[VanderSP]

Okay, i´ve found what happens ->

the bloom alpha was finally fixed on 137.5 the bloom alpha fix was broken in 137.3

I´ve tested in 137.2 and 136.0 it worked (pasting the file on node_modules)

it´s funny when something breaks an old thing fix just before the fix release :D

Also, i wonder what setClearColor does? when i got it working with i described above or not, setClearColor does nothing always.

[Mugen87]

I've decided to having a look at this issue since the PR #23356 does not work for me. Here is my branch with the suggested change from above: https://github.com/Mugen87/three.js/commit/551e7b5640b76e0e966b40e1cf9b2a1cb7122e95

UnrealBloomPass now supports transparent backgrounds however the bloom does not blend correctly with the HTML background image. The same artifacts appear like mentioned in https://github.com/mrdoob/three.js/issues/14104#issuecomment-478351541 (see the three screenshots at the end of the post).

You can easily verify this by opening the following URL and set the bloom strength and radius to maximum.

https://raw.githack.com/Mugen87/three.js/dev60/examples/webgl_postprocessing_unreal_bloom.html

Looking at the suggested fix, a simple enhancement in the fragment shader is not sufficient.

[luca-viggiani]

Hi, I've been using this tweak with 128 and worked for long time very well. Today I switched to 142 and the tweak doesn't work anymore. I looked at comment above and saw that the issue should be fixed in 137.5 so I removed the tweak from my code but it still renders a black background with 142. Here is my code:

const BLOOM_PARAMS = {
    resolution: new THREE.Vector2(1024, 1024),
    strength: 2,
    radius: .65,
    threshold: .85
}

const CAMERA_PARAMS = {
    fov: 50,
    pixelRatio: window.devicePixelRatio,
    nearClippingPlane: .1,
    farClippingPlane: 10000
}

this.#scene = new THREE.Scene();
this.#renderer = new THREE.WebGLRenderer({
    antialias: true,
    alpha: true,
    format: THREE.RGBAFormat,
    logarithmicDepthBuffer: true
});

this.#renderer.shadowMap.type = THREE.PCFSoftShadowMap;
this.#renderer.outputEncoding = THREE.sRGBEncoding;
this.#renderer.setPixelRatio(window.devicePixelRatio);
this.#renderer.setClearColor(0x000000, 0.0);

this.#shadowRoot.appendChild(this.#renderer.domElement);

this.#camera = new THREE.PerspectiveCamera(...Object.values(CAMERA_PARAMS));

this.#passes = {
    render: new RenderPass(this.#scene, this.#camera),
    bloom: new UnrealBloomPass(...Object.values(BLOOM_PARAMS))
}

this.#effectComposer = new EffectComposer(this.#renderer);
Object.values(this.#passes).forEach(pass => this.#effectComposer.addPass(pass));

Where I do wrong? Thanks!

[Mugen87]

saw that the issue should be fixed in 137.5

The issue is actually not fixed. Sorry for the confusion in this thread.

Besides, I don't think the issue can be properly fixed since there is no way to correctly additively blend the bloom with a HTML background.

I think we should close the issue and state that UnrealBloomPass requires a non-transparent scene background.

[lviggiani]

saw that the issue should be fixed in 137.5

The issue is actually not fixed. Sorry for the confusion in this issue.

Besides, I don't think the issue can be properly fixed since there is no way to correctly additively blend the bloom with a HTML background.

I think we should close the issue and state that UnrealBloomPass requires a non-transparent scene background.

Thanks for the feedback! With 128 + the tweak it actually works and I get transparent background as desired. Unfortunately it stop working with more recent releases. Mixing 142 and 128 for the UnrealBloomPass maybe is not a good idea but I'll give it a try

[lviggiani]

Ok, the mix doesn't work. I'll stick to 128 + frag shader for now

[CountAsWun]

Ah i was so hopeful when i saw that this was fixed but maybe we are back at square one. I don't know much about modifying this library and the underlying WebGL tech but i've spent 10 hours trying to get background transparency working as it is essential to my use case.

What changed from the previous versions that now break background transparency?

What is it about RenderPass that handles background transparency with ease that breaks in UnrealBloom?

I don't have the background to fully understand how to implement this, but the solution here seems simple as this allows bloom to work with WebGL, I just need to understand how to implement it with threejs.UnrealBloom

https://stackoverflow.com/questions/53422916/how-to-draw-transparent-background-in-webgl-on-canvas