KhronosGroup / glTF-Blender-IO

Blender glTF 2.0 importer and exporter
https://docs.blender.org/manual/en/latest/addons/import_export/scene_gltf2.html
Apache License 2.0
1.48k stars 317 forks source link

Blender 4.2 [222d] key not found error Khronos 4.2.28 #2246

Closed ronh991 closed 4 months ago

ronh991 commented 4 months ago

Describe the bug exporting to gltf using MSFS exporter - error in

File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_sampler.py", line 21, in gather_sampler group_path.append(bpy.data.materials[tab[0]])


KeyError: 'bpy_prop_collection[key]: key "" not found'

This is after the MSFS program uses the gather_material_hook

**To Reproduce**
Steps to reproduce the behavior:
open blend file export to gltf - with MSFS exporter extension
Does not error out without added program to trigger gather_material_hook.  This did not happen with Khronos  4.2.23 happens now in 4.2.28

**Expected behavior**
not understanding what tab[0] is but should check for None ""

**Screenshots**
None
[test_42_error.zip](https://github.com/KhronosGroup/glTF-Blender-IO/files/15393122/test_42_error.zip)

**.blend file/ .gltf (mandatory)**
A zipped folder containing a .blend file for exporter issues and a .gltf file for importer issues. Please take care to include all resources, such as textures in the .zip file.
If you are not allowed to share your file, make sure to create a new one from scratch that shows the issue. You can also try to delete or modify some data (objects, change textures, replace some meshes by some placeholder), to be able to share it without restriction.

**Version**
 - OS: win 10
 - Blender Version 4.2.0 222d

**Additional context**
Add any other context about the problem here.

FULL ERROR

```
Traceback (most recent call last):
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\io\exp\gltf2_io_user_extensions.py", line 14, in export_user_extensions
    hook(*args, export_settings)
  File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\4.2\extensions\user_default\gltf_exporter_extension\io\msfs_export.py", line 145, in gather_material_hook
    MSFSMaterial.export(gltf2_material, blender_material, export_settings)
  File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\4.2\extensions\user_default\gltf_exporter_extension\io\msfs_material.py", line 281, in export
    extension.to_extension(blender_material, gltf2_material, export_settings)
  File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\4.2\extensions\user_default\gltf_exporter_extension\com\msfs_material_props.py", line 1243, in to_extension
    result["detailColorTexture"] = MSFSMaterial.export_image(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\4.2\extensions\user_default\gltf_exporter_extension\io\msfs_material.py", line 199, in export_image
    texture_info = gather_texture_info(
                   ^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture_info.py", line 30, in gather_texture_info
    return __gather_texture_info_helper(primary_socket, blender_shader_sockets, 'DEFAULT', filter_type, export_settings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_cache.py", line 37, in wrapper_cached
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture_info.py", line 70, in __gather_texture_info_helper
    index, factor, udim_image = __gather_index(blender_shader_sockets, None, export_settings)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture_info.py", line 226, in __gather_index
    return gltf2_blender_gather_texture.gather_texture(blender_shader_sockets, use_tile, export_settings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_cache.py", line 37, in wrapper_cached
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture.py", line 45, in gather_texture
    sampler=__gather_sampler(blender_shader_sockets, export_settings),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture.py", line 200, in __gather_sampler
    return gather_sampler(
           ^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_cache.py", line 37, in wrapper_cached
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_sampler.py", line 21, in gather_sampler
    group_path.append(bpy.data.materials[tab[0]])
                      ~~~~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'bpy_prop_collection[key]: key "" not found'
```
julienduroure commented 4 months ago

Hello, This is a duplicate of #2231, will be probably fixed by #2232

ronh991 commented 4 months ago

I am still having issues with this error. Also why does the export (without the MSFS addon) seem to go through the model parts three time each?

4.2.39

10:09:59 | INFO: Starting glTF 2.0 export
10:09:59 | INFO: Starting glTF 2.0 export
10:09:59 | INFO: Starting glTF 2.0 export
10:09:59 | INFO: Extracting primitive: part085
10:09:59 | INFO: Extracting primitive: part085
10:09:59 | INFO: Extracting primitive: part085
10:09:59 | WARNING: More than one shader node tex image used for a texture. The resulting glTF sampler will behave like the first shader node tex image.
10:09:59 | WARNING: More than one shader node tex image used for a texture. The resulting glTF sampler will behave like the first shader node tex image.
10:09:59 | WARNING: More than one shader node tex image used for a texture. The resulting glTF sampler will behave like the first shader node tex image.
10:09:59 | WARNING: More than one shader node tex image used for a texture. The resulting glTF sampler will behave like the first shader node tex image.
10:09:59 | WARNING: More than one shader node tex image used for a texture. The resulting glTF sampler will behave like the first shader node tex image.
10:09:59 | WARNING: More than one shader node tex image used for a texture. The resulting glTF sampler will behave like the first shader node tex image.
10:09:59 | WARNING: The active Vertex Color will not be exported, as it is not used in the node tree of the material
10:09:59 | WARNING: The active Vertex Color will not be exported, as it is not used in the node tree of the material
10:09:59 | WARNING: The active Vertex Color will not be exported, as it is not used in the node tree of the material
10:09:59 | INFO: Primitives created: 1
10:09:59 | INFO: Primitives created: 1
10:09:59 | INFO: Primitives created: 1
10:10:00 | INFO: Finished glTF 2.0 export in 0.7435505390167236 s
10:10:00 | INFO: Finished glTF 2.0 export in 0.7435505390167236 s
10:10:00 | INFO: Finished glTF 2.0 export in 0.7435505390167236 s

test_42_error_03.zip

ronh991 commented 4 months ago

followup - it has something to do with color attributes. test_42_error.zip

julienduroure commented 4 months ago

Please pack your textures, I can't reproduce without any textures

ronh991 commented 4 months ago

File size becomes too big > 25 - can you accept a link to google drive?

julienduroure commented 4 months ago

Yes

julienduroure commented 4 months ago

(and three time warnings are linked to a bad initialization of logging system. When there was a crash, the logging is not correctly purged, neither purged at initialization)

ronh991 commented 4 months ago

Here is a trimmed down blend file. This error happens with the MSFS enabled. I understand if you cannot help.

The error happens when I have a hook to gather_material. I was trying to get around a earlier error.

Here is the hook code


    def gather_material_hook(self, gltf2_material, blender_material, export_settings):
        # blender 4.0 issue with detail textures added twice once correct and once as a regular basecolor texture
        # if it has a detail color texture then set basecolor texture to none
        #print("gather_material_hook - Started with gltf2_material", gltf2_material, gltf2_material.pbr_metallic_roughness, gltf2_material.pbr_metallic_roughness.base_color_texture)
        #print("gather_material_hook - blender material - delete base color before", blender_material, blender_material.msfs_detail_color_texture, blender_material.msfs_base_color_texture)
        if blender_material.msfs_detail_color_texture is not None and blender_material.msfs_base_color_texture is None:
            #blender_material.msfs_material_type == "msfs_windshield":
            gltf2_material.pbr_metallic_roughness.base_color_texture = None
            print("*** MSFS WARNING *** - blender material - delete the base color", blender_material, blender_material.msfs_detail_color_texture, blender_material.msfs_base_color_texture)
        #print("gather_material_hook - blender material - delete base color after", blender_material, blender_material.msfs_detail_color_texture, blender_material.msfs_base_color_texture)

        # Apr 25,2024 update to exporter may have fixed this issue and this code is not needed

        # # blender 3.3 removes base color values with base color texture - have to add back in
        # #print("gather_material_hook - Started with gltf2_material", gltf2_material, gltf2_material.pbr_metallic_roughness, gltf2_material.pbr_metallic_roughness.base_color_texture, gltf2_material.pbr_metallic_roughness.base_color_factor)
        # base_color = blender_material.msfs_base_color_factor
        # gltf2_base_color = gltf2_material.pbr_metallic_roughness.base_color_factor
        # #print("gather_material_hook - blender material - set base color factor before", blender_material, blender_material.msfs_base_color_texture, base_color[0], base_color[1], base_color[2], base_color[3], gltf2_base_color)
        # if base_color is not None and gltf2_base_color is None:
            # print("*** MSFS WARNING *** - changing base_color_factor because none", blender_material, base_color[0], base_color[1], base_color[2], base_color[3])
            # gltf2_material.pbr_metallic_roughness.base_color_factor = [base_color[0],base_color[1],base_color[2],base_color[3]]
        # if gltf2_base_color is not None:
            # if not equality_check(base_color, gltf2_base_color, len(base_color), len(gltf2_base_color)):
                # print("*** MSFS WARNING *** - changing base_color_factor because different in node", blender_material, base_color, gltf2_base_color)
                # if base_color[0] == 1.0 and base_color[1] == 1.0 and base_color[2] == 1.0 and base_color[3] == 1.0:
                    # gltf2_material.pbr_metallic_roughness.base_color_factor = gltf2_base_color
                # else:
                    # gltf2_material.pbr_metallic_roughness.base_color_factor = [base_color[0],base_color[1],base_color[2],base_color[3]]
        # #print("gather_material_hook - blender material - set base color after", blender_material, gltf2_material.pbr_metallic_roughness.base_color_factor)

        if self.properties.enable_msfs_extension:
            #print("gather_material_hook - export")
            MSFSMaterial.export(gltf2_material, blender_material, export_settings)
        #print("gather_material_hook - Done")

https://drive.google.com/file/d/1SI4kpBTy_DRYRd5L1MrbStbqyAlqhp6a/view?usp=sharing

ronh991 commented 4 months ago

current erro

gather_image_hook - Done
Traceback (most recent call last):
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\io\exp\gltf2_io_user_extensions.py", line 14, in export_user_extensions
    hook(*args, export_settings)
  File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\4.2\extensions\user_default\gltf_exporter_extension\io\msfs_export.py", line 145, in gather_material_hook
    MSFSMaterial.export(gltf2_material, blender_material, export_settings)
  File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\4.2\extensions\user_default\gltf_exporter_extension\io\msfs_material.py", line 281, in export
    extension.to_extension(blender_material, gltf2_material, export_settings)
  File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\4.2\extensions\user_default\gltf_exporter_extension\com\msfs_material_props.py", line 1243, in to_extension
    result["detailColorTexture"] = MSFSMaterial.export_image(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\4.2\extensions\user_default\gltf_exporter_extension\io\msfs_material.py", line 199, in export_image
    texture_info = gather_texture_info(
                   ^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture_info.py", line 30, in gather_texture_info
    return __gather_texture_info_helper(primary_socket, blender_shader_sockets, 'DEFAULT', filter_type, export_settings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_cache.py", line 37, in wrapper_cached
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture_info.py", line 70, in __gather_texture_info_helper
    index, factor, udim_image = __gather_index(blender_shader_sockets, None, export_settings)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture_info.py", line 226, in __gather_index
    return gltf2_blender_gather_texture.gather_texture(blender_shader_sockets, use_tile, export_settings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_cache.py", line 37, in wrapper_cached
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture.py", line 45, in gather_texture
    sampler=__gather_sampler(blender_shader_sockets, export_settings),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\material\gltf2_blender_gather_texture.py", line 200, in __gather_sampler
    return gather_sampler(
           ^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_cache.py", line 37, in wrapper_cached
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "F:\Downloads\blender-4.2.0\4.2\scripts\addons_core\io_scene_gltf2\blender\exp\gltf2_blender_gather_sampler.py", line 21, in gather_sampler
    group_path.append(bpy.data.materials[tab[0]])
                      ~~~~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'bpy_prop_collection[key]: key "" not found'
ronh991 commented 4 months ago

Ok this looks like a me problem. Have to understand why I can't have two base color type nodes, or two ORM nodes, or two normal nodes. Your texture_info class and texture_gather image_gather are causing me my problems.

ronh991 commented 4 months ago

any chance of getting a hook

gather_texture_info_hook_before at the beginning of the __gather_texture_info_helper

ronh991 commented 4 months ago

my findings. Just FYI __gather_sampler is failing on not matching

id(mat.node_tree) == id(first_valid_shader_node.group_path[0].original)

so the group_path_str is blank or null

julienduroure commented 4 months ago

Did you reproduce it with glTF export (not MSFS)?

ronh991 commented 4 months ago

no I cannot - I just added numerous print statements that lead me to it, using the blend file above.

*** MSFS WARNING *** - gather_image_hook - Started with  <io_scene_gltf2.io.com.gltf2_io.Image object at 0x000001F7A4C82150> (<bl_ext.user_default.gltf_exporter_extension.io.msfs_material.NodeSocket object at 0x000001F79C0DCB10>,)
*** MSFS WARNING *** gather_image_hook - image GLASS_DetailMap01_mask - Copy image/png <io_scene_gltf2.io.exp.gltf2_io_image_data.ImageData object at 0x000001F7A4C82190>
*** MSFS WARNING *** - gather_image_hook - Done
******************************* MSFS Here 1b *************
******************************* MSFS Here 2b shader nodes ************* [<io_scene_gltf2.blender.exp.material.gltf2_blender_search_node_tree.NodeTreeSearchResult object at 0x000001F79C0D6550>]
******************************* MSFS Here 3b first_valid_shader_node >0 ************* [bpy.data.materials['WINDSHIELD BUCKER BBB_int']] <bpy_struct, Material("WINDSHIELD BUCKER BBB_int") at 0x000001F79BD13288>
******************************* MSFS Here 3b first_valid_shader_node >0 - mat ************* <bpy_struct, Material("Dots Stroke") at 0x000001F79BD3BD08> Dots Stroke False 140715229186832 2162986556160
******************************* MSFS Here 3b first_valid_shader_node >0 - mat ************* <bpy_struct, Material("material24") at 0x000001F79BD3BB88> material24 True 2162986647808 2162986556160
******************************* MSFS Here 3b first_valid_shader_node >0 - mat ************* <bpy_struct, Material("WINDSHIELD BUCKER BBB_int") at 0x000001F79BD13288> WINDSHIELD BUCKER BBB_int True 2162986646976 2162986556160
******************************* MSFS Here 3b path *************
*** MSFS WARNING *** - MSFSMaterial - Base Color ERROR
MSFSMaterial - export_image DEFAULT if - return (None, {}, {}, None)
AsoboMaterialDetail - to extension normal <bpy_struct, Image("FROST_DETAIL_NORM.PNG.png") at 0x000001F79BC8E408>
MSFSMaterial - export_image type NORMAL
MSFSMaterial - export_image NORMAL if
ronh991 commented 4 months ago
def __gather_sampler(blender_shader_sockets, export_settings):
    shader_nodes = [get_texture_node_from_socket(socket, export_settings) for socket in blender_shader_sockets]
    if len(shader_nodes) > 1:
        export_settings['log'].warning(
            "More than one shader node tex image used for a texture. "
            "The resulting glTF sampler will behave like the first shader node tex image."
        )
    print("******************************* MSFS Here 1b *************")
    first_valid_shader_node = next(filter(lambda x: x is not None, shader_nodes))

    # group_path can't be a list, so transform it to str
    print("******************************* MSFS Here 2b shader nodes *************", shader_nodes)

    sep_item = "##~~gltf-sep~~##"
    sep_inside_item = "##~~gltf-inside-sep~~##"
    group_path_str = ""
    if len(first_valid_shader_node.group_path) > 0:
        print("******************************* MSFS Here 3b first_valid_shader_node >0 *************", first_valid_shader_node.group_path, first_valid_shader_node.group_path[0].original)
        # Retrieving the blender material using this shader tree
        for mat in bpy.data.materials:
            print("******************************* MSFS Here 3b first_valid_shader_node >0 - mat *************", mat, mat.name, mat.use_nodes, id(mat.node_tree), id(first_valid_shader_node.group_path[0].original))
            if mat.use_nodes is True and id(mat.node_tree) == id(first_valid_shader_node.group_path[0].original):
                group_path_str += mat.name  # TODO if linked, we can have multiple materials with same name...
                break
    if len(first_valid_shader_node.group_path) > 1:
        for idx, i in enumerate(first_valid_shader_node.group_path[1:]):
            group_path_str += sep_item
            if idx == 0:
                group_path_str += first_valid_shader_node.group_path[0].name
            else:
                group_path_str += i.id_data.name
            group_path_str += sep_inside_item
            group_path_str += i.name
    print("******************************* MSFS Here 3b path *************", group_path_str)

    return gather_sampler(
        first_valid_shader_node.shader_node,
        group_path_str,
        export_settings)
ronh991 commented 4 months ago

I am "fixing" the original code for Blender 4.2 the code adds a fake BDSF node and links it to the image node just to get the texture _info data. I'm sure this is not the right way, but it's what I have to work with

    @staticmethod
    def export_image(
        blender_material, blender_image, type, export_settings, normal_scale=None
    ):
        nodes = blender_material.node_tree.nodes
        links = blender_material.node_tree.links

        # should all be in a try finally to remove Fake nodes
        # Fake Fake Fake Fake Fake Fake Fake Fake Fake Fake 

        # Create a fake texture node temporarily (unfortunately this is the only solid way of doing this)
        texture_node = nodes.new("ShaderNodeTexImage")
        texture_node.name = "FakeTextureNode"
        texture_node.image = blender_image

        # Create shader to plug texture into
        shader_node = nodes.new("ShaderNodeBsdfPrincipled")
        shader_node.name = "FakeBSDFNode"
        print("MSFSMaterial - export_image type", type)

        # Gather texture info
        if type == "DEFAULT":
            link = links.new(shader_node.inputs["Base Color"], texture_node.outputs[0])
            # from Khronos gltf core example pbr sockets gather_texture_info

            print("MSFSMaterial - export_image DEFAULT if", link)
            base_color_socket = get_socket_msfs(blender_material, "Base Color", "FakeBSDFNode")
            #alpha_socket = get_socket_msfs(blender_material, "Alpha", "FakeBSDFNode")

            # keep sockets that have some texture : color and/or alpha
            print("MSFSMaterial - socket", base_color_socket)
            inputs = tuple(
                socket for socket in [base_color_socket]
            )
            print("MSFSMaterial - inputs", inputs)

            # had to add in a default socket argument for compatability with 4.0  added (),
            # had to remove the default socket argument for compatability with 4.1+  removed (),
            print("MSFSMaterial - export_image DEFAULT if - sockets", inputs[0], inputs)
            for inp in inputs:
                print("MSFSMaterial - inputs list", inp)
            try:

                # errors at this line
                texture_info = gather_texture_info(
                    inputs[0],
                    inputs,
                    export_settings,
                    #shader_node.inputs["Base Color"],
                    #(shader_node.inputs["Base Color"],),
                    #export_settings,
                )
            except:
                print("*** MSFS WARNING *** - MSFSMaterial - Base Color ERROR")
                texture_info = None, {}, {}, None
            finally:
                pass
            print("MSFSMaterial - export_image DEFAULT if - return", texture_info)
ronh991 commented 4 months ago

fixed on my end - I was looping through material rather than material.node_tree.