Incorrect alpha when rendering meshes with blur and many small polygons

[unixpickle]

🐛 Bugs / Unexpected behaviors

I am finding that I cannot use higher values of sigma like 1e-4 when my meshes are high resolution (i.e. have a lot of polygons). For the example below, I create a sphere with marching cubes, and then try to render it. Using the default recommended sigma of 1e-4 results in a broken image.

Please advise on whether this is expected behavior, why it's happening, or if there's something I should be doing to mitigate it.

With sigma=1e-4:

With much smaller sigma=1e-5:

Instructions To Reproduce the Issue:

Here is an example script to render the above images:

from pytorch3d.structures import Meshes
from pytorch3d.renderer import (
    BlendParams,
    DirectionalLights,
    FoVPerspectiveCameras,
    MeshRasterizer,
    MeshRenderer,
    RasterizationSettings,
    SoftPhongShader,
    TexturesVertex,
)
import skimage
import torch
import numpy as np
from PIL import Image

device = torch.device('cuda')

# Create a field representing a sphere
resolution = 128
xs = np.linspace(-1, 1, num=resolution)
field = np.array([[[1.0 - np.sqrt(x**2 + y**2 + z**2) for z in xs] for y in xs] for x in xs])

# Create a mesh from the above
verts, faces, _, _ = skimage.measure.marching_cubes(field, 0)
verts = (verts / (resolution / 2) - 1.0)
old_f1 = faces[:, 0].copy()
faces[:, 0] = faces[:, 1]
faces[:, 1] = old_f1

meshes = Meshes(
    verts=[torch.from_numpy(verts.copy()).cuda()],
    faces=[torch.from_numpy(faces.copy()).cuda()],
)
meshes.textures = TexturesVertex(
    verts_features=torch.tensor([[1.0, 0.0, 0.0]], device=device).repeat(len(verts), 1)[None],
)

x = [1.0, 0.0, 0.0]
y = [0.0, 0.0, -1.0]
z = [0.0, -1.0, 0.0]
origin = [0.0, 3.0, 0.0]
R = np.array([x, y, z])
cameras = FoVPerspectiveCameras(
    R=R.T[None],
    T=-(R.T @ origin)[None],
    fov=90,
    degrees=True,
    device=device,
)

lights = DirectionalLights(
    ambient_color=((0.2, 0.2, 0.2),),
    diffuse_color=((0.7, 0.7, 0.7),),
    specular_color=((0.1, 0.1, 0.1),),
    direction=((0, np.sqrt(2), -np.sqrt(2)),),
    device=device,
)

sigma = 1e-4

raster_settings_soft = RasterizationSettings(
    image_size=512,
    blur_radius=np.log(1.0 / 1e-4 - 1.0) * sigma,
    faces_per_pixel=50,
    max_faces_per_bin=100000,
    bin_size=None,
    perspective_correct=False,
)
renderer = MeshRenderer(
    rasterizer=MeshRasterizer(cameras=cameras, raster_settings=raster_settings_soft),
    shader=SoftPhongShader(
        device=meshes.device,
        cameras=cameras,
        lights=lights,
        blend_params=BlendParams(sigma=sigma, background_color=(0, 0, 0)),
    ),
)
result = renderer(meshes)

# The below code renders the result as an image.
img = np.clip(np.round(result[0].cpu().numpy() * 255), 0, 255).astype(np.uint8)
display(Image.fromarray(img))

[bottler]

High sigma makes the face edges be less well defined. But you shouldn't need overlap of faces to cover every internal pixel if the mesh is watertight, so I'm not sure what is happening. It's worth checking if you get the same with bin_size=0.