is it possible to user EffectComposer / ShaderPass with the Viewer renderer?

[jordaneast1]

Hi,

Thanks for the repo, it's super cool!! I'm building a portfolio website in React and have been implementing this, it looks great :)

I'm wondering if it's possible to use the GausianSplat3D.Viewer alongside the normal three.js postprocessing pipeline?

Havent had much luck with the standard practice of attaching the renderer to the EffectComposer and using composer.render() instead of renderer.render() ..... ( or in this case viewer.render() ? )

Thanks in advance!!

[mkkellogg]

I haven't had too much direct experience with using the effect composer, but others have with some success. I think you're going to have to avoid using the selfDrivenMode so that you control rendering, and you would then call the render() function on the effect composer, but not on the viewer. Something like:

const renderWidth = 800;
const renderHeight = 600;

const rootElement = document.createElement('div');
rootElement.style.width = renderWidth + 'px';
rootElement.style.height = renderHeight + 'px';
document.body.appendChild(rootElement);

const renderer = new THREE.WebGLRenderer({
    antialias: false
});
renderer.setSize(renderWidth, renderHeight);
rootElement.appendChild(renderer.domElement);

const threeScene = new THREE.Scene();

const composer = new EffectComposer(renderer);
const scenePass = new RenderPass(threeScene, camera);
const splatPass = new RenderPass(viewer.splatMesh, camera);
const outlinePass = new OutlinePass(new THREE.Vector2(window.innerWidth, window.innerHeight), scene, camera, selectedObjects);
outlinePass.renderToScreen = true;
outlinePass.selectedObjects = selectedObjects;

composer.addPass(scenePass);
composer.addPass(splatPass);
composer.addPass(outlinePass);

const camera = new THREE.PerspectiveCamera(65, renderWidth / renderHeight, 0.1, 500);
camera.position.copy(new THREE.Vector3().fromArray([-1, -4, 6]));
camera.up = new THREE.Vector3().fromArray([0, -1, -0.6]).normalize();
camera.lookAt(new THREE.Vector3().fromArray([0, 4, -0]));

const viewer = new GaussianSplats3D.Viewer({
    'selfDrivenMode': false,
    'renderer': renderer,
    'camera': camera,
    'selfDrivenMode': false
});
viewer.addSplatScene('<path to .ply, .ksplat, or .splat file>')
.then(() => {
    requestAnimationFrame(update);
});

function update() {
    requestAnimationFrame(update);
    viewer.update();
    composer.render()
}

I haven't tried the above code, but it should be close :)

[jordaneast1]

awesome!!

I was missing const splatPass = new RenderPass(viewer.splatMesh, camera); and composer.addPass(splatPass);

works great now :)

Thanks so much!!

[jordaneast1]

apologies, one more question - the splatPass seems to only render the splatMesh and not other objects or backgrounds in the scene, and does not take into account the depth buffer so overwrites the other scenePass.

Do you know how I might include scene object/background, or perhaps include alpha in the viewer render?

I'm also learning the effects composer so if you have any idea that would be greatly appreciated!! I will experiment with a few things and report back if i find anything.

[mkkellogg]

Would you be willing to share your code? Maybe I could help debug.