Point cloud, transparency, and depthTest

[nvaytet]

Hi,

I am trying to create a point cloud with a custom shader that allows me to change the opacity of certain points. I am using this to render an opaque slice inside a 3d volume, surrounded by lower opacity points. While I have managed to create the custom shader, I am facing an issue with the transparency and what appears to be the order in which the points are drawn.

My code example is the following:

import pythreejs as p3
from matplotlib import cm
import numpy as np

x = np.arange(50.) / 50.
x, y, z = np.meshgrid(x, x, x)
# Purposefully split the points in two halves to reveal the split at the center of the x axis
positions1 = np.array([x.flatten(), y.flatten(), z.flatten()]).astype('float32').T
positions2 = np.array([-x.flatten(), y.flatten(), z.flatten()]).astype('float32').T
positions = np.concatenate([positions1, positions2])

cmap = cm.get_cmap('viridis')
scalar_map = cm.ScalarMappable(cmap=cmap)
colors = scalar_map.to_rgba(np.sin(positions[:, 2])).astype('float32')

alpha = np.where(np.abs(positions[:, 0]) < 0.1, 1.0, 0.03)
colors[:, 3] = alpha

geometry = p3.BufferGeometry(
    attributes={
        'position': p3.BufferAttribute(array=positions),
        'rgba_color': p3.BufferAttribute(array=colors)
    })

pixel_size = 0.02

material = p3.ShaderMaterial(
            vertexShader='''
precision highp float;
attribute vec4 rgba_color;
varying vec3 mypos;
varying vec4 vColor;
varying vec4 projected;
float xDelta, yDelta, zDelta, delta;
void main(){
    vColor = rgba_color;
    gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
    // use the delta between the point position and camera position for size
    xDelta = (position[0]-cameraPosition[0]) * (position[0]-cameraPosition[0]);
    yDelta = (position[1]-cameraPosition[1]) * (position[1]-cameraPosition[1]);
    zDelta = (position[2]-cameraPosition[2]) * (position[2]-cameraPosition[2]);
    delta = pow(xDelta + yDelta + zDelta, 0.5);
    gl_PointSize = %f / delta;
}
''' % (580.0 * pixel_size, ),
            fragmentShader='''
precision highp float;
varying vec4 vColor;
void main() {
    gl_FragColor = vColor;
}
''',
            vertexColors='VertexColors',
            transparent=True,
            depthTest=True,
        )

points = p3.Points(geometry=geometry, material=material)
view_width = 800
view_height = 500
camera = p3.PerspectiveCamera(position=[2.0, 0, 0], aspect=view_width/view_height)
scene = p3.Scene(children=[points, camera], background="#DDDDDD")
controller = p3.OrbitControls(controlling=camera)
renderer = p3.Renderer(camera=camera, scene=scene, controls=[controller],
                    width=view_width, height=view_height)
renderer

With depthTest=True in the ShaderMaterial, the scene looks good from one viewing angle

But when I move the camera to view the scene from below, the low opacity points start to block some of the high opacity points which are then no longer visible behind

I believe this has to do with the way threejs and Webgl work, where it performs a depthTest and does not render points that would be hidden behind other points along the line of sight (to save on computational cost), and does not take into account transparency.

So I tried to set depthTest=False, and while this generally works better with transparent points, it leads to strange artifacts that are related to the order in which the points are drawn. So in my opaque slab, depending on the viewing angle, I may be seeing some points from the back of the slab in front of other points that should be at the front. This leads to the strange 'rift' that can be seen in the centre of the slab, where I have purposefully drawn the left half of the points after the right half of the points to highlight the issue:

I have tried to read up on threejs, transparency and depthTest but so far I have not found a fix for my problem. Many thanks for any help!

[vidartf]

Since there has been no answers here, I'm going to close this, as it doesn't seem to be about an issue in pythreejs itself, but rather a question on how to resolve certain problems in webgl/threejs. If there is any questions/problems directly related to pythreejs, please leave a comment and I can reopen the issue.