Latest three.js

[andrewvarga]

Hi! This is a great library!

I'm having a bit of trouble using it with the latest three.js, is that something you're planning to add support for? thanks!

[lucas-jones]

Sure, I've upgraded this project to 161

Commit https://github.com/lucas-jones/three-lightmap-baker/commit/0df3df7e0fd5b30c4f96d7813c9d33c5affec617

[andrewvarga]

Thank you for the quick answer!

For me the vertex shader was not compiling:

ERROR: 0:86: 'uv2' : redefinition

but if I removed the attribute vec2 uv2; line it compiled. I wonder why it is different for you vs me?

I'm trying to integrate this into three.js editor, and the result with the Avocado sample model is something like this after I apply the lightmap (and set channel to 2 as in your code):

It looks like it's using the wrong UV channel?

[lucas-jones]

You might need to set the textures channel to 2 (not quite sure how in Threejs Editor)

I had to change this in code too https://github.com/lucas-jones/three-lightmap-baker/commit/0df3df7e0fd5b30c4f96d7813c9d33c5affec617#diff-dba681e72df960f9fff3b45c7a5700c9a7b4bf9e9d218ffacb910cf0bb454556R296

Also the lightmapping process will generate & build UV2. You might need to ensure these are added to the model

How did you export the lightmap?

[andrewvarga]

You might need to set the textures channel to 2 (not quite sure how in Threejs Editor)

I had to change this in code too 0df3df7#diff-dba681e72df960f9fff3b45c7a5700c9a7b4bf9e9d218ffacb910cf0bb454556R296

Also the lightmapping process will generate & build UV2. You might need to ensure these are added to the model

How did you export the lightmap?

Sorry it took some time to test a few things out..

if all I do is clone your repo and try the demo with the official Avocado.glb I get this error: [.WebGL-0x12400c26900] GL_INVALID_OPERATION: Vertex buffer is not big enough for the draw call https://github.com/KhronosGroup/glTF-Sample-Models/tree/main/2.0/Avocado/glTF-Binary

I found this is because this model has a tangent attribute which is not handled by xatlas-three probably, so adding this code helped after the packAtlas call:

await unwrapper.packAtlas(geometry, 'uv2', 'uv');
// fix for tangents
for (const g of geometry) {
   if (g.hasAttribute('tangent')) {
      g.deleteAttribute('tangent');
      g.computeTangents();
   }
}

Lightmap export: I just added a function that reads the texture into a canvas and displays it in the dom:

function debugTexture(renderer, renderTarget) {
    const width = renderTarget.width;
    const height = renderTarget.height;
    const pixels = new Float32Array(width * height * 4); // 4 components (RGBA) per pixel, as floats
    renderer.readRenderTargetPixels(renderTarget, 0, 0, width, height, pixels);

    // Step 3: Convert the floating point data to 8-bit data for visualization
    const displayPixels = new Uint8Array(width * height * 4);
    for (let i = 0; i < pixels.length; i++) {
        // Assuming the float data is in the range [0, 1], scale to [0, 255]
        // Adjust this as necessary based on how your data is scaled
        displayPixels[i] = Math.min(255, Math.max(0, Math.floor(255 * pixels[i])));
    }

    // Continue as before to create a data texture, canvas, and image
    const dataTexture = new THREE.DataTexture(displayPixels, width, height, THREE.RGBAFormat);
    dataTexture.needsUpdate = true;

    const canvas = document.createElement('canvas');
    canvas.width = width;
    canvas.height = height;
    const context = canvas.getContext('2d');

    const imgData = context.createImageData(width, height);
    imgData.data.set(displayPixels);
    context.putImageData(imgData, 0, 0);

    debugImage = debugImage || new Image();
    debugImage.src = canvas.toDataURL();
    debugImage.style.zIndex = 999999;
    debugImage.style.top = '300px';
    debugImage.style.position = 'absolute';

    if (!debugImage.parentNode) {
        document.body.appendChild(debugImage);
    }
}

[andrewvarga]

The channel seems to be correct now, there are a few seaming artifacts on the lightmap, I wonder if those can be improved somehow?

[lucas-jones]

This will be due to light leaking - the lightmapper trying to sample points that are placed inside the model rather than the surface. Good article on it here

You could render at a higher resolution and then downscale the lightmap manually - might help remove the artifacts.

This repo is more of a proof of concept than an actual tool

If you're after a proper tool

• You can bake Indirect Light / Ambient Occlusion using Blender
  • There is also a Addon for Blender for lightmap baking - The Lightmapper

[andrewvarga]

• The Lightmapper

Thank you that is useful! Do you think there are any parameters I could tweak that could help this either in lightmapperOptions or at the unwrap stage? xatlas-three has a lot of parameters, but I couldn't find one that resolved this for me..

 unwrapper.chartOptions = {
    fixWinding: true,
    maxBoundaryLength: 0,
    maxChartArea: 0,
    maxCost: 2,
    maxIterations: 1,
    normalDeviationWeight: 2,
    normalSeamWeight: 4,
    roundnessWeight: 0.009999999776482582,
    straightnessWeight: 6,
    textureSeamWeight: 0.1,
    useInputMeshUvs: false,
};
unwrapper.packOptions = {
    bilinear: true,
    blockAlign: false,
    bruteForce: false,
    createImage: false,
    maxChartSize: 0,
    padding: 0,
    resolution: 1024,
    rotateCharts: true,
    rotateChartsToAxis: true,
    texelsPerUnit: 0
};

In my particular case I'm mostly interested in the ambient occlusion part, and I think the black lines are coming from the texels which are "in between" the faces of the unwrapped geometry, so I wonder if it would be possible that the "lightmap" (more like an occlusion map in this case) defaults to being white and becomes darker when there are more hits by bvhIntersectFirstHit..maybe somehow detecting when the current pixel doesn't belong to any unwrapped triangle in the geometry and make that white..?

[gkjohnson]

there are a few seaming artifacts on the lightmap, I wonder if those can be improved somehow?

These are caused by the color transition at the edge of the UV islands causing the sample to transition to black when sampling due to linear interpolation. There are no UV modifications that can be made to address this and simply changing the background color will just change the color of the seam.

Instead the best approach is to inflate the color of the UV islands into the uncolored background area so there are no interpolation artifacts at the edges. Of course these things still break down with mipmapping (regular texturing does, as well) but in typical viewing scenarios it should be okay. You can read more about it in this Edge Padding Polygon article. As far as I know there's no out-of-the-box solution to generating this in the three.js ecosystem.

[lucas-jones]

Nice article on edge padding

Some dilation is done when rendering the texture atlas (world positions & normals) - in the hope it would pad out a few pixels to prevents this

https://github.com/lucas-jones/three-lightmap-baker/blob/main/src/atlas/renderAtlas.ts#L125-L131

[gkjohnson]

Some dilation is done when rendering the texture atlas (world positions & normals) - in the hope it would pad out a few pixels

It looks like that offset value shifts every uv point consistently (ie everything will be shifted down to the right, for example), which would likely exacerbate the problem. Even if you expanded the UV islands "outward" so an apparent padding was added this would still cause issues since the positions used to sample light will no longer be aligned (at least slightly) to the same positions used when sampling the texture.

I think the best solution is a post-processing of the generated light map to flood the background with data / color from the rendered uv islands.

[andrewvarga]

there are a few seaming artifacts on the lightmap, I wonder if those can be improved somehow?

These are caused by the color transition at the edge of the UV islands causing the sample to transition to black when sampling due to linear interpolation. There are no UV modifications that can be made to address this and simply changing the background color will just change the color of the seam.

Instead the best approach is to inflate the color of the UV islands into the uncolored background area so there are no interpolation artifacts at the edges. Of course these things still break down with mipmapping (regular texturing does, as well) but in typical viewing scenarios it should be okay. You can read more about it in this Edge Padding Polygon article. As far as I know there's no out-of-the-box solution to generating this in the three.js ecosystem.

Thanks! Does that mean that if I set the textures filtering to NEAREST, those artifacts should be gone (of course the lightmap will become pixelated which is a bigger problem, so this is not a solution, I just want to verify that this is the problem).

Maybe I don't understand this in all detail, but are there any pixels in the position/normal texture that don't belong to any geometry in the unwrapped UV set? So these would be pixels that would not be covered by any triangles if you draw the geometry to this texture in UV space. I was just thinking if there are such pixels, it would be an easy way to have those texels white instead of black..

As a quick post processing hack, I also tried tried to just make fully black pixels -> fully white. Note t So from this (where you can clearly see some areas of the texture that don't belong to any triangles on the mesh, eg. the bottom left corner):

To this:

Note that I'm using this texture as an occlusion map, not a lightmap (even though technically it's a lightmap, I only use the ambient light. Surprisingly this removed those small black line artifacts and there's nothing wrong that I see..I'm thinking it's because the area of the mesh which would be fully black isn't really visible in 3D in this particular model..

[gkjohnson]

Thanks! Does that mean that if I set the textures filtering to NEAREST, those artifacts should be gone

No, it does not. Textures are limited in resolution so pixels do not exactly align with the edges of triangles.

are there any pixels in the position/normal texture that don't belong to any geometry in the unwrapped UV set

Of course, these are the black pixels outside of the UV islands.

I was just thinking if there are such pixels, it would be an easy way to have those texels white instead of black..

You're just shifting the problem. The seams will be now be white instead of black. It may not be as noticeable in the avocado case but any seams in darker corners will now just be white. If you want this to work in the general case you need to inflate the color from the islands out to the edges of texture as the link I shared demonstrates.

[andrewvarga]

Thank you, that makes sense, I'll give color inflation a try!

I have tried the lightmapper with multiple models and have a few problems with models like this:

this is the generated lightmap:

This is the original model and the generated glb (lightmap added to occlusion map sofa.zip

Do you have any ideas what this could be caused by? Maybe problems in the original mesh geometry / topology? It seems on the lightmap there are some diagonal "stretches" that seemingly overlap large areas covering multiple islands..

[lucas-jones]

The model looks fine before the exports. Looks like the buffer attributes are messed up somehow (index buffer might be wrong?)

generateAtlas will run unwrapper.PackAtlas REF

Which then uses the xatlas-three library to convert the geometry into ones that support UV2 (can change the attributes from the original) REF

Or it could even be with the way your exporting the GLB