[lightmap] Add “lightmap” component

[donmccurdy]

Lightmap could be added through a separate component, like envMap. Adding directly to “material” doesn’t make sense, since it can’t be applied to models with existing materials.

[biomorphica]

Some reasons to add light maps directly to the material 1) avoid pixelation from overly dense map packing. If you had to use a single texture atlas for all objects in the scene, it would surely get crowded and lead to a pixellation problem 2) scene organization: keep lights maps associated with other textures. They are generated in 3d apps (max maya blender etc), on a per material basis, in a process called Render To Texture (aka 'baked lighting'). It makes sense to keep them in the material as they are composited over other textures (normal, roughness, etc). The problem is that light maps must use a second uv map channel as the baking fragments the texture. Light maps are a bit different that AO, which could get away with using a single map for the entire scene as detail is not so relevant. Light maps have the base color textures with lighting added in, so detail is crucial. Part of the art of light mapping is generating high resolultion maps where needed (areas of game play focus) and low res for peripheral objects. The 'problem' is accessing the second map channel. Not sure if gltf exports multiple uv map channels, but if it doesn't, it should (I think it does). THREE.js enforces lightmaps and AO to use map chan 2, and all others map chan 1. This would be ideal. It is simply a matter of AFrame parsing the glb, understanding which texture is the light map, and assigning and reading map chan 2. I suggest using the emissive slot in gltf as the light channel.
Of course one can use lightmaps now without any new changes, by placing the renderToTexture (ie the lightmap) in the baseColor or emissive map slot. The problems is that this light map can't be used with other textures, because it requires another uv set. So really all we need is the ability to use 2 uv sets. Light maps are standard fare for Unity and Unreal, and are really important for building realism or sophisticated looks through texturing. Ultimately you'd want to harness the power of pbr in addition to light maps, so that you could have normal maps, roughness, metallic (and all the power of substance designer) in addition to baked lighting. Obviously you don't want light maps for objects that move, but for static object, it really transforms the flat look to something complex and emotional. I think light maps are important tool for any game designer, and a big step to take AFrame beyond the "webgl look" and bring it up to the shading sophistication of Unity.

[biomorphica]

Here is an example of what light maps do for a scene.

without light maps http://www.sensorium.love/experiments/yamashiro/walkthroughaf/yamawalkthrough.html

with light maps http://www.sensorium.love/experiments/yamashiro/walkthroughlit2/yamawalkthrough.html

[biomorphica]

Here is a brilliant example of blending light maps with the complexity of shaders. Not sure if this was done with AFrame, but this is what we need. (would love to have ;) https://showroom.littleworkshop.fr/

[donmccurdy]

Light maps have the base color textures with lighting added in, so detail is crucial.

I don't follow this – lightmaps are independent of the base color texture, I think?

Part of the art of light mapping is generating high resolultion maps where needed (areas of game play focus) and low res for peripheral objects.

Quality is, in the end, a matter of UV space. Like with AO, you can allocate more or less UV space to a region, regardless of the number and size of textures involved. This is why both AO and Lightmaps often use a second UV channel, distinct from the UV channel used for the base color texture and normals.

Not sure if gltf exports multiple uv map channels...

It does, yes.

I suggest using the emissive slot in gltf as the light channel...

Even though glTF does not have a lightmap feature yet, once it is parsed to a threejs or a-frame material, the .lightMap texture slot is available. The proposal in this PR is to add a new component that allows a lightmap to be added to a glTF model, rather than bundled into it. I don't want to treat an emissive map as a lightmap in a shared library, although users applications are free to do things like that.

For example:

<a-entity gltf-model="src: scene.glb"
  lightmap="src: lightmap.png"></a-entity>

This should be a fairly simple component to create, but would perhaps not handle (or at least not handle easily) cases where the model requires multiple lightmaps. That can be avoided by packing them in the texturing application, but in the end the best workflow would be for glTF to support lightmaps itself. This has been proposed, and feedback/encouragement on https://github.com/KhronosGroup/glTF/pull/1017 would be welcome.

[biomorphica]

Great that this is moving forward!

Light maps have the base color textures with lighting added in, so detail is crucial.

I don't follow this – lightmaps are independent of the base color texture, I think?

 yes, light maps are independent of base color, in three.js.  What I

meant here is when the light maps are generated (in the 3D app, ie max), the render adds the base color to the lighting render pass, in the renderer. Apologies for the confusion.

but would perhaps not handle (or at least not handle easily) cases where the model requires multiple lightmaps.

 Just to be clear, inside gltf (see below) the mesh object contains the

primitive object, and it is on the primitive level that you assign texcoord channel, and materials, and thus the lightmap. The light map should be able to be unique for each primitive (ie sharing the same ligth map among primitives should not be mandatory). So long as you do it that way, everyone will be happy, and no problems should arise. "meshes": [ { "primitives": [ { "attributes": { "POSITION": 1, "NORMAL": 2, "TEXCOORD_0": 3 }, "indices": 0, "mode": 4, "material": 0 } ], "name": "BonsaiBendELeafLiteR_05" }, { "primitives": [ { "attributes": { "POSITION": 5, "NORMAL": 6, "TEXCOORD_0": 7 }, "indices": 4, "mode": 4, "material": 0 } ], "name": "BonsaiBendELeafLiteR_04"

On Sat, Jul 20, 2019 at 2:29 PM Don McCurdy notifications@github.com wrote:

Light maps have the base color textures with lighting added in, so detail is crucial.

I don't follow this – lightmaps are independent of the base color texture, I think?

Part of the art of light mapping is generating high resolultion maps where needed (areas of game play focus) and low res for peripheral objects.

Size of the texture is, in the end, a matter of UV space. Like with AO, you can allocate more or less UV space to a region, regardless of the number and size of textures involved. This is why both AO and Lightmaps often use a second UV channel, distinct from the UV channel used for the base color texture and normals.

Not sure if gltf exports multiple uv map channels...

It does, yes.

I suggest using the emissive slot in gltf as the light channel...

Even though glTF does not have a lightmap feature yet, once it is parsed to a threejs or a-frame material, the .lightMap texture slot is available. The proposal in this PR is to add a new component that allows a lightmap to be added to a glTF model, rather than bundled into it. I don't want to treat an emissive map as a lightmap in a shared library, although users applications are free to do things like that.

For example:

<a-entity gltf-model="src: scene.glb"

lightmap="src: lightmap.glb">

This should be a fairly simple component to create, but would perhaps not handle (or at least not handle easily) cases where the model requires multiple lightmaps. That can be avoided by packing them in the texturing application, but in the end the best workflow would be for glTF to support lightmaps itself. This has been proposed, and feedback/encouragement on KhronosGroup/glTF#1017 https://github.com/KhronosGroup/glTF/pull/1017 would be welcome.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/donmccurdy/aframe-extras/issues/279?email_source=notifications&email_token=ACEFEOAX566HF7PQDA3ZTNDQAN7TTA5CNFSM4GDLF7IKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NWLDI#issuecomment-513500557, or mute the thread https://github.com/notifications/unsubscribe-auth/ACEFEOHNAPO5ZSHAHTF2BKLQAN7TTANCNFSM4GDLF7IA .

[donmccurdy]

Note that this issue is for the creation of an A-Frame component that would let you load a lightmap from a texture file, or perhaps multiple lightmaps, that is separate from your glTF model. Having lightmaps embedded in glTF models would require the glTF specification to support this, tracked elsewhere, and not in scope for A-Frame Extras to solve. Best-case scenario, then, you'd need to know the name of each primitive to assign the lightmap:

<a-entity gltf-model="src: scene.glb"
  lightmap_top="src: lightmap_top.png"
  lightmap_base="src: lightmap_base.png"></a-entity>

^This would imply that primitives named "top" and "base" exist in the GLB file.

[biomorphica]

The most important part of this is the ability to have a 2nd texcoord channel. I presume that the new AFrame component PR would provide that. I am coming to realize that manipulating the object3D of an element is where all the power lies. Often materials don't come in as you would like, and need to be tweaked, and this can only be done by traversing the object3D, finding the material and setting its parameters. I am just learning to do this, and it is a bit frustrating, but I can see how important this is. I could probably take care of all my light map issues once I master this skill. In the back of my mind I keep thinking that a component that allowed you to navigate a gltf entity, much like the inspector in the scene, would be hugely useful. Sadly the inspector does not expose the inside of the object3D, but the DOM does. So I realize it is up to me to master javascript to gain this ability. Just thinking out loud about what would be useful to a casual (ie artist) aframe user.

On Sat, Jul 20, 2019 at 3:09 PM Don McCurdy notifications@github.com wrote:

Note that this issue is for the creation of an A-Frame component that would let you load a lightmap from a texture file, or perhaps multiple lightmaps, that is separate from your glTF model. Having lightmaps embedded in glTF models would require the glTF specification to support this, tracked elsewhere. Best-case scenario, then, you'd need to know the name of each primitive to assign the lightmap:

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/donmccurdy/aframe-extras/issues/279?email_source=notifications&email_token=ACEFEOAZFKL5UVKJ6JCYG7TQAOEK5A5CNFSM4GDLF7IKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NW3ZI#issuecomment-513502693, or mute the thread https://github.com/notifications/unsubscribe-auth/ACEFEOHLMEIGHEYWHXNVFFTQAOEK5ANCNFSM4GDLF7IA .

[biomorphica]

Also, not sure if you saw this, but I sent you a mail a while ago about a grid component replacement for the aframe-extras.grid https://www.npmjs.com/package/aframe-extras.grid The grid in that package is simply a plane with texture, which, as they say in Hollywood, looks unprofessional. Not parametric, not transparent, and vulnerable to z-fighting. I made a grid component https://glitch.com/~grid-component that renders with the line object, so it solves all of these problems. Feel free to use this instead if you want. I don't need or want credit. No worries if you don't have time. Just letting you to know about it.

On Sat, Jul 20, 2019 at 3:36 PM Thomas Williams biomorphica@gmail.com wrote:

The most important part of this is the ability to have a 2nd texcoord channel. I presume that the new AFrame component PR would provide that. I am coming to realize that manipulating the object3D of an element is where all the power lies. Often materials don't come in as you would like, and need to be tweaked, and this can only be done by traversing the object3D, finding the material and setting its parameters. I am just learning to do this, and it is a bit frustrating, but I can see how important this is. I could probably take care of all my light map issues once I master this skill. In the back of my mind I keep thinking that a component that allowed you to navigate a gltf entity, much like the inspector in the scene, would be hugely useful. Sadly the inspector does not expose the inside of the object3D, but the DOM does. So I realize it is up to me to master javascript to gain this ability. Just thinking out loud about what would be useful to a casual (ie artist) aframe user.

On Sat, Jul 20, 2019 at 3:09 PM Don McCurdy notifications@github.com wrote:

Note that this issue is for the creation of an A-Frame component that would let you load a lightmap from a texture file, or perhaps multiple lightmaps, that is separate from your glTF model. Having lightmaps embedded in glTF models would require the glTF specification to support this, tracked elsewhere. Best-case scenario, then, you'd need to know the name of each primitive to assign the lightmap:

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/donmccurdy/aframe-extras/issues/279?email_source=notifications&email_token=ACEFEOAZFKL5UVKJ6JCYG7TQAOEK5A5CNFSM4GDLF7IKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NW3ZI#issuecomment-513502693, or mute the thread https://github.com/notifications/unsubscribe-auth/ACEFEOHLMEIGHEYWHXNVFFTQAOEK5ANCNFSM4GDLF7IA .

[colinfizgig]

I’ve written a light map component. It’s here... https://github.com/colinfizgig/aframe_Components

The only downside right now was that I was having trouble targeting preexisting Second UV sets. So currently it just duplicates the first UV set. If the light map matches the UVs it works. This isn’t optimal, since the lightmap could me optimized for packing. Donmccurdy do you know of a way to target the second map? It may be that the second UVs are not exported with GLTF?

Either way hopefully this component helps.

[donmccurdy]

A glTF file might or might not include a second UV set. While threejs and A-Frame require it for AO and lightmaps, glTF does not. If you just have one UV set in Blender, and export to glTF, you'll just have one UV set in the resulting file. You can duplicate the UVs in Blender first, or duplicate them after loading in your component:

if ( material.lightMap
    && geometry.attributes.uv2 === undefined
    && geometry.attributes.uv !== undefined ) {

  geometry.addAttribute( 'uv2', new THREE.BufferAttribute( geometry.attributes.uv.array, 2 ) );

}

[biomorphica]

Thanks for sharing this! I'll test it out when I have some time.

On Tue, Oct 1, 2019 at 2:59 PM Don McCurdy notifications@github.com wrote:

A glTF file might or might not include a second UV set. While threejs and A-Frame require it for AO and lightmaps, glTF does not. If you just have one UV set in Blender, and export to glTF, you'll just have one UV set in the resulting file. You can duplicate the UVs in Blender first, or duplicate them after loading in your component:

if ( material.lightMap && geometry.attributes.uv2 === undefined && geometry.attributes.uv !== undefined ) {

geometry.addAttribute( 'uv2', new THREE.BufferAttribute( geometry.attributes.uv.array, 2 ) );

}

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/donmccurdy/aframe-extras/issues/279?email_source=notifications&email_token=ACEFEODPLW73C6PDB4W2IFDQMPB3XA5CNFSM4GDLF7IKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAC4Y5I#issuecomment-537250933, or mute the thread https://github.com/notifications/unsubscribe-auth/ACEFEOAOO7NEBXJJNL64M5LQMPB3XANCNFSM4GDLF7IA .