Headless mode possible?

[sweco-sekrsv]

Your toolset looks awesome. Would it be possible running it headless mode? Something like this: blender --background --python run_The_Lightmapper.py

Essentially I would like the script to import a gltf/obj file then do all of the magic that your tools do (unwrapping, baking etc) and then export a new gltf/obj file. I'll probably consume the output in babylon.js

[Naxela]

Hi, thanks for trying out The Lightmapper

What you describe should be possible, as that's what the background bake mode does under the hood:

https://github.com/Naxela/The_Lightmapper/blob/fcc235628627d26a77d5f846f2a10addd93fde29/addon/utility/build.py#L143

As for the code inside the python background file, I haven't tried, but I imagine something like the following could work (if modified to work):

import bpy, thelightmapper

#Import your gltf/obj file using something like bpy.ops.import_scene_gltf()

#Do some adjustsments here if you want, like setting quality, denoise options, encoding (8-bit png or EXR for babylon), filtering, etc.

#Select all objects in the file and mark them to be baked
for obj in bpy.data.objects:
   if obj.type == "Mesh":
      obj.TLM_ObjectProperties.tlm_mesh_lightmap_use = True

#Begin baking
#Use thelightmapper.addon.utility.build.prepare_build(0, True) for background baking,
#as bpy.ops.tlm_build_lightmaps() operator throws a context error (no context in background)
thelightmapper.addon.utility.build.prepare_build(0, True);

#Save the file again, maybe export the unwrapped gltf back first?
bpy.ops.save_as_mainfile()

As mentioned I don't really have any insight in working with gltf/glb or babylon, but I think something like the above should work some of the way. By the way, I recommend using the 0.4 branch (https://github.com/Naxela/The_Lightmapper/tree/0.4) as I've recently fixed some background-related bugs, and I'm planning on adding network rendering soon, which will also improve on the background-baking process.

Hope it helps, and feel free to ask if you've any questions!

[sweco-sekrsv]

Cool. I will give it a shot!

Would it be possible to also support RGBD output? This is what babylon.js can consume easily

I also have anoher question regarding altas textures. If I have many objects in the scene to be baked, is it also possible to make one or several atlas-textures automatically? For example if I define a max atlas texture of 2048x2048, all of the objects in the scene would be packed into one atlas texture, if there is not enough area in the first atlas texture it should create another one.

[Naxela]

Hi, let me know if you run into trouble!

I'm already planning on writing some documentation on transfering the lightmaps to Godot and Three.JS, I'm thinking I might as well add Babylon.JS to my to do list as well.

As for RGBD output, I actually had that implemented a while back (last year or so), but I removed the option as I thought no one would use it, I only just noticed that RGBD was added recently to Babylon.js - In any case, reimplementing RGBD output again would be relatively simple.

As for the atlas textures, currently it packs all your objects automatically upon baking, so all the object UV's are forced together regardless in the prepacked mode. It sounds like what you're seeking is the postpack mode, where the predefined object lightmaps with their respective resolutions is packed into one larger Atlas? It's what I'm planning on adding for the postpack mode:

[sweco-sekrsv]

Yes, you can see this thread in the babylon.js forum where they are using RGBD encoded textures for the lightmaps: https://forum.babylonjs.com/t/lightmaps-not-bright-enough/12649/4

My plan is also to try to use basis universal textures for the lightmaps (https://github.com/BinomialLLC/basis_universal), as KTX2 files. This will be a good optimization if the content in the RGBD (or maybe the 32 bit hdr textures) information translates well into that format. A tool that can both optimize the gltf file as well as convert textures to this format is gltfpack (https://github.com/zeux/meshoptimizer/tree/master/gltf)

Yeah, I think the Post-packaging mode is what I'm looking for. I will check back to see your progress on that mode.

Will let you know if I run into problems or if I find some smart things along the way.

[Naxela]

Hi,

I've made some progress in this regard, RGBD has been implemented again and post-packing AtlasGroups has also been implemented (although overflow haven't been added yet), but there's a percentage utilization counter, which should at least make it possible to do it manually for now without too much hassle: https://github.com/Naxela/The_Lightmapper/releases/tag/0.4.3.4

As for the KTX2 files, it seems that I might be able to integrate the binaries in some way as an encoding option by using the command line compression tool. I'll put it on my todo list to investigate more

[sweco-sekrsv]

Nice!

I tried a simple workflow running from cmd-line just to get started and it seems to be working. However I do run into a problem. After
thelightmapper.utility.build.prepare_build(0, True) I get the message "Lightmap building finished" and then Blender just quits with no error message. At this stage textures have been created and looks ok!

After the thelightmapper is done I want to export a gltf file.

thelightmapper.utility.build.prepare_build(0, True)
print('This will never print')
bpy.ops.export_scene.gltf(filepath="F:\\blender_baking\\testbake.gltf",export_texcoords=True, export_format = 'GLTF_SEPARATE')

No gltf file is being created and the print message wont show.

[Naxela]

Hi,

I found out that this was caused why a sys.exit() command that I had in place from when it's running in background from within Blender, and as a safety-kill for threads, but I've added a default parameter which should prevent that from happening unless it's called (https://github.com/Naxela/The_Lightmapper/commit/32e0fc6ba26045b97a3b3ed2c1c49d41de1a7156)

Your code example should work now

[sweco-sekrsv]

Hi,

I found out that this was caused why a sys.exit() command that I had in place from when it's running in background from within Blender, and as a safety-kill for threads, but I've added a default parameter which should prevent that from happening unless it's called (32e0fc6)

Your code example should work now

Great, it works now!

[sweco-sekrsv]

Hi!

Would it be possible to add a mode that skips creating/assigning the new lightmap to materials when the bake is done?

After baking/unwrapping is done I would like to export the unwrapped result to a new gltf file but skip adding the lightmap textures to the gltf file since that does not work very well. The reason it does not work is that it seems that the lightmap textures is being assigned as the main texture but I also need to keep the original textures intact. So instead I will sideload the lightmaps in babylon.

Does that make sense?

[Naxela]

Hi,

If I understood you correctly, you're looking for the "Don't apply materials" function, which does pretty much that and leaves the materials as they were before baking, but adds a link to the object's lightmaps as a custom property (object["Lightmap"]):

Would this work?

[sweco-sekrsv]

Ahh, missed that. Thats great!

I have another question regarding the rgbd texture output. The rgbd textures from thelightmapper becomes overbright in babylon.js Check one exameple in the attached zip-file

If I create the rgdb texture from the original hdr texture using this playground: https://playground.babylonjs.com/#95Y4D7

They look good in babylon.js. Are there any more settings I can use when encoding to rgbd from thelightmapper?

rgdb_problem.zip

[Naxela]

Hi,

I looked into the shader code, and it seems the RGBD encoding babylon uses might differ a little bit from mine, it seems it has something to do with the maxrange (the range of which the float values are compressed into), it's seems that code from here (https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/ShadersInclude/helperFunctions.fx) at the function toRGBD is hardcoded to 255, where as I think mine is inverse of that - Can you try setting encoding range to something else (maybe even just 1 or 255) to see if it works?:

Otherwise, I'm planning on porting my RGBM and RGBD encoding to offscreen GPU shaders anyway (like LogLuv) is, since it's faster, but it will probably take a little time, and I can more or less use the same shader that Babylon uses

[sweco-sekrsv]

Hi!

I tried changing the encoding range to 1 and also 255 but neither of them worked unfortunatly. Hopefullt it will work nicely when you port it to GPU shaders.

[sweco-sekrsv]

I tried the latest 0.4.4.3 WIP Release with the rgbd gpu encoding options. It seems to work good when running from within Blender, the rgbd textures now loads fine in babylon.js. Thats great! However in background mode I get the following error.

Info: Baking map saved to internal image, save it externally or pack it
F:\work\blender_baking\testout_auto\room.floor.001_baked.hdr
F:\work\blender_baking\testout_auto\room.walls.inner.001_baked.hdr
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "F:\work\blender_baking\blender-2.90.0-windows64\example-bake_lightmapper.py", line 117, in <module>
    thelightmapper.utility.build.prepare_build(0, True)
  File "F:\work\blender_baking\blender-2.90.0-windows64\2.90\scripts\addons\thelightmapper\addon\utility\build.py", line 113, in prepare_build
    begin_build()
  File "F:\work\blender_baking\blender-2.90.0-windows64\2.90\scripts\addons\thelightmapper\addon\utility\build.py", line 575, in begin_build
    encoding.encodeImageRGBDGPU(img, sceneProperties.tlm_encoding_range, dirpath, 0)
  File "F:\work\blender_baking\blender-2.90.0-windows64\2.90\scripts\addons\thelightmapper\addon\utility\encoding.py", line 150, in encodeImageRGBDGPU
    offscreen = gpu.types.GPUOffScreen(input_image.size[0], input_image.size[1])
SystemError: GPU functions for drawing are not available in background mode

Blender quit

[Naxela]

Hi,

Sadly, even though I managed to get GPU encoding to work, I stumpled upon the same limitations in Blender, namely that GPU functions aren't available in background mode, so I have to get CPU encoding to work anyway, which I'll try to start working on this week.

[Naxela]

I think I might have gotten the CPU encoding to work (which should also work in background mode), and the results seem similar from what I tested in https://github.com/Naxela/The_Lightmapper/commit/207d7ba8749d6ec2963c9dcc576e96666aff32ac

[sweco-sekrsv]

Great! It seems to work fine with the RGBD-CPU encoding, I also get very similar results compared to GPU-encoding. Would it be possible to clamp intensity values in the encoded output? Maybe with a tweakable GUI setting. In direct light areas I get to high intensity (floor texture is getting washed out) in the rgdb lightmap. Look at this image:

It also looks like there is a bit more reddish/brownish tint in the lightmap which is not visible in the rendered image from blender.

[Naxela]

Hi, good to hear that it works!

I can probably add an option to clamp the values eventually, although I think a more "correct" approach would be to use tonemapping postprocessing. I'm not entirely sure about how babylon.js does it's tonemapping, although from a quick search it seems to be possible:

https://doc.babylonjs.com/how_to/how_to_use_postprocesses#tonemap https://doc.babylonjs.com/api/classes/babylon.imageprocessingconfiguration#tonemapping_aces

You'll want to set it to ACES (Academy standard, also sometimes referred to as filmic) if possible, and adjust the exposure manually (if you're not using some kind of eye adaption feature). This might also affect the color tint issues you're having. I think the standard babylon.js tonemapper would be comparable to the old Blender tonemapper (Standard, linear sRGB):

In any case, I'll look into manually clamping exposure as a feature

[sweco-sekrsv]

You were correct, by applying a tonemapping_aces in the ImageProcessingConfiguration in babylon both the reddish tint and over exposed direct light areas were fixed. Now the result look very close to the Blender rendering. Thank you for your speedy response, really appreciated!

[Naxela]

No problem, happy to help in terms of progress and having the addon coming to good use with external engines. I'll keep this issue open until I've added the option to clamp HDR float values

[OmiBit97]

Hi there Naxela, first of all congratulations for the plugin you have developed, it is very useful. Although I have a problem that I can't solve. I have created an object with multi material and lights and when I make the bake, it doesn´t apply the brick texture or cobblestones into the UV Map export. How can I export a lightmap UV (red circle) with the correct texture instead of a white lightmap only?

Thank you very much.

[Naxela]

@OmiBit97 Hi, by default the lightmapper only exports the lighting data to a 32-bit HDR file, but it exact procedure varies a bit depending on the result you want. If I understand you correctly, you want a texture map that has both lightdata + texture data combined? - In that case, I think what you want can be achieved by using the "Complete map mode":

This is one of the areas that I haven't documented that well in the Lightmapper, but in general the difference between combined and complete is that combined bakes only lighting (combines direct + indirect), where complete bakes everything (including texture).