Configuration file not found for node type: bnw_spots_3

[yuyingyeh]

As title, I'd like to run the test script. It shows that the configuration file for node type: bnw_spots_3 for graph concrete_raw_panels.sbs is missing. Any helps will be appreciated! Thanks!

python3 test_sampler.py sbs/match_v1/concrete_raw_panels.sbs -r test_diffmat
Traceback (most recent call last):
  File "test_sampler.py", line 99, in <module>
    main()
  File "test_sampler.py", line 72, in main
    toolkit_path=args.toolkit_path)
  File "/usr/local/lib/python3.7/dist-packages/diffmat/translator/graph_trans.py", line 46, in __init__
    super().__init__(root)
  File "/usr/local/lib/python3.7/dist-packages/diffmat/translator/base.py", line 168, in __init__
    self._init_node_translators()
  File "/usr/local/lib/python3.7/dist-packages/diffmat/translator/graph_trans.py", line 194, in _init_node_translators
    node_config = load_node_config(node_type)
  File "/usr/local/lib/python3.7/dist-packages/diffmat/translator/util.py", line 332, in load_node_config
    raise FileNotFoundError(f'Configuration file not found for {mode} type: '
FileNotFoundError: Configuration file not found for node type: bnw_spots_3

[Polar1s]

Hi Yu-Ying,

This error happens when DiffMat tries to translate a generator node (e.g., bnw_spots_3) and fails. This is expected since DiffMat doesn't implement any generator node in the current version. To circumvent this error, add an -e option to the test script to generate input textures using Substance Designer command-line tools.

We noticed that neither README.md nor the source file adequately referenced this error. We will make necessary improvements to documentation in the next release. For now, please regard the -e option as mandatory.

Thanks!

[yuyingyeh]

Hi Beichen,

Thanks for your reply! I have tried the same command with -e option: python3 test_sampler.py sbs/match_v1/concrete_raw_panels.sbs -e -t /opt/Adobe/Adobe_Substance_3D_Designer -r test_diffmat

but encounter this error

Traceback (most recent call last):
  File "test_sampler.py", line 99, in <module>
    main()
  File "test_sampler.py", line 95, in main
    img_format=img_format)
  File "/usr/local/lib/python3.7/dist-packages/diffmat/optim/sampler.py", line 229, in evaluate
    maps = graph.evaluate_maps()
  File "/usr/local/lib/python3.7/dist-packages/diffmat/core/graph.py", line 156, in evaluate_maps
    memory.update({key: val for key, val in zip(inst['result'], result) if key})
TypeError: zip argument #2 must support iteration

[Polar1s]

Unfortunately, I wasn't able to reproduce the error on my Ubuntu 18.04 (SD version 11.2.2) or Windows 10 (SD version 12.1.1) system. Could you tell me the Substance Designer version you're using?

[Polar1s]

In addition, identifying which node triggered the error also helps. You could add a -l verbose option to enable full-scale logging, where diffmat will time each node execution. The problematic node is after the last successful timing entry.

[yuyingyeh]

I'm using version 12.1.1 on Ubuntu 18.04. Actually here is the node info coming out before the error message:

[INFO] Compiled material graph program (51 nodes):
  bnw_spots_3_0: () -> bnw_spots_3_0_bnw_spots_3
  uniform_0: () -> uniform_0
  grunge_map_013_0: () -> grunge_map_013_0_output
  uniform_1: () -> uniform_1
  clouds_3_0: () -> clouds_3_0_clouds_3
  fractal_sum_base_0: () -> fractal_sum_base_0_fractal_sum_base
  uniform_2: () -> uniform_2
  fractal_sum_base_1: () -> fractal_sum_base_1_fractal_sum_base
  grunge_map_007_0: () -> grunge_map_007_0_output
  tile_generator_0: () -> tile_generator_0_tilegenerator
  gradient_0: (bnw_spots_3_0_bnw_spots_3) -> gradient_0
  curvature_smooth_0: (uniform_1) -> curvature_smooth_0
  blur_2: (clouds_3_0_clouds_3) -> blur_2
  levels_7: (clouds_3_0_clouds_3) -> levels_7
  blur_0: (fractal_sum_base_0_fractal_sum_base) -> blur_0
  slope_blur_grayscale_2: (fractal_sum_base_0_fractal_sum_base, clouds_3_0_clouds_3) -> slope_blur_grayscale_2
  levels_5: (fractal_sum_base_1_fractal_sum_base) -> levels_5
  safe_transform_grayscale_0: (grunge_map_007_0_output) -> safe_transform_grayscale_0
  blur_1: (tile_generator_0_tilegenerator) -> blur_1
  slope_blur_0: (gradient_0, grunge_map_007_0_output) -> slope_blur_0
  levels_1: (curvature_smooth_0) -> levels_1
  levels_3: (curvature_smooth_0) -> levels_3
  blend_0: (bnw_spots_3_0_bnw_spots_3, levels_7, levels_5) -> blend_0
  safe_transform_grayscale_1: (levels_5) -> safe_transform_grayscale_1
  blend_1: (bnw_spots_3_0_bnw_spots_3, safe_transform_grayscale_0, None) -> blend_1
  slope_blur_grayscale_3: (blur_1, blur_1) -> slope_blur_grayscale_3
  levels_4: (slope_blur_0) -> levels_4
  blend_4: (uniform_2, uniform_2, levels_3) -> blend_4
  blend_12: (levels_3, uniform_0, None) -> blend_12
  blend_10: (slope_blur_grayscale_2, blend_0, safe_transform_grayscale_1) -> blend_10
  histogram_scan_0: (blend_1) -> histogram_scan_0
  levels_2: (slope_blur_grayscale_3) -> levels_2
  blend_8: (grunge_map_013_0_output, blend_10, None) -> blend_8
  blend_7: (histogram_scan_0, levels_1, None) -> blend_7
  slope_blur_grayscale_1: (levels_2, blur_2) -> slope_blur_grayscale_1
  blend_2: (grunge_map_007_0_output, blend_8, None) -> blend_2
  blend_5: (levels_4, blend_4, blend_7) -> blend_5
  blend_9: (blend_7, blend_12, None) -> blend_9
  slope_blur_grayscale_0: (slope_blur_grayscale_1, blur_0) -> slope_blur_grayscale_0
  hsl_0: (blend_5) -> hsl_0
  sharpen_0: (blend_9) -> sharpen_0
  levels_6: (slope_blur_grayscale_0) -> levels_6
  levels_0: (hsl_0) -> levels_0
  invert_grayscale_0: (levels_6) -> invert_grayscale_0
  basecolor: (levels_0) -> basecolor
  blend_3: (bnw_spots_3_0_bnw_spots_3, invert_grayscale_0, None) -> blend_3
  blend_6: (blend_3, slope_blur_grayscale_1, None) -> blend_6
  blend_11: (blend_6, blend_2, None) -> blend_11
  normal_0: (blend_11) -> normal_0
  normal: (normal_0) -> normal
  roughness: (sharpen_0) -> roughness
[INFO] Graph name: concrete_raw_panels  #nodes: 51  #params: 2138

I have tried -l verbose but nothing changed. It seems that the format of result has some issues here...? https://github.com/mit-gfx/diffmat/blob/bd7a8d5d0e500e6b583af30697e82a518103e522/diffmat/core/graph.py#L156

[Polar1s]

I have elevated this issue to a known bug in the system.

If no [DEBUG] message follows the screen output above, my best guess is that input textures were not correctly generated or read, so the result of the first node is None (hence not iterable). What you can do is check the test_diffmat/concrete_raw_panels/external_input/default directory for generated input textures. The error should have come from reading if the following files are present.

[Polar1s]

Also, does the problem occur to other graphs (*.sbs files) you have tested?

[yuyingyeh]

I have elevated this issue to a known bug in the system.

If no [DEBUG] message follows the screen output above, my best guess is that input textures were not correctly generated or read, so the result of the first node is None (hence not iterable). What you can do is check the test_diffmat/concrete_raw_panels/external_input/default directory for generated input textures. The error should have come from reading if the following files are present.

I only found concrete_raw_panels_input.sbs in test_diffmat/concrete_raw_panels/external_input/default. It seems that the input textures are not generated in my case.

[yuyingyeh]

Also, does the problem occur to other graphs (*.sbs files) you have tested?

I have tried aluminium_cut.sbs, but the same issue happened.

[yuyingyeh]

I have elevated this issue to a known bug in the system. If no [DEBUG] message follows the screen output above, my best guess is that input textures were not correctly generated or read, so the result of the first node is None (hence not iterable). What you can do is check the test_diffmat/concrete_raw_panels/external_input/default directory for generated input textures. The error should have come from reading if the following files are present.

I only found concrete_raw_panels_input.sbs in test_diffmat/concrete_raw_panels/external_input/default. It seems that the input textures are not generated in my case.

For aluminium_cut.sbs, there is a single line [DEBUG] message after [INFO] message:

[INFO] (diffmat.core - sampler.py) Graph name: aluminium_cut  #nodes: 29  #params: 46
[DEBUG] (diffmat.core - base.py) Node uniform_0: 2.852 ms

[Polar1s]

It makes sense that the issue repeats on other graphs if it is related to the input texture generator itself.

Did you see any warning or error messages from the SD executables? Diffmat typically invokes SD command-line tools (sbscooker and sbsrender) to generate input textures in the external_input/default folder. In your case, it seems unusual to produce neither input textures nor warning/error messages.

Nonetheless, you could try manually generating the textures. Enter the test_diffmat/concrete_raw_panels/external_input/default folder and run the following shell command:

/opt/Adobe/Adobe_Substance_3D_Designer/sbscooker concrete_raw_panels_input.sbs --output-path "."
/opt/Adobe/Adobe_Substance_3D_Designer/sbsrender render concrete_raw_panels_input.sbsar --output-format png --output-name "{outputNodeName}" --output-path "."

Let me know if these commands generate the textures without warning/error messages.

[yuyingyeh]

It makes sense that the issue repeats on other graphs if it is related to the input texture generator itself.

Did you see any warning or error messages from the SD executables? Diffmat typically invokes SD command-line tools (sbscooker and sbsrender) to generate input textures in the external_input/default folder. In your case, it seems unusual to produce neither input textures nor warning/error messages.

Nonetheless, you could try manually generating the textures. Enter the test_diffmat/concrete_raw_panels/external_input/default folder and run the following shell command:

/opt/Adobe/Adobe_Substance_3D_Designer/sbscooker concrete_raw_panels_input.sbs --output-path "."
/opt/Adobe/Adobe_Substance_3D_Designer/sbsrender render concrete_raw_panels_input.sbsar --output-format png --output-name "{outputNodeName}" --output-path "."

Let me know if these commands generate the textures without warning/error messages.

I have tried the two commands. The first one works but not for the second one with version 11.2.2 and 12.1.1 on Ubuntu. Both commands work with version 12.1.1 on MacOS.

Since Adobe only release .rpm version for Linux, I need to convert it to .deb with alien. I'm not sure if something went wrong during this conversion or any license information required during the installation. Basically I installed the SD with commands:

sudo alien Adobe_Substance_3D_Designer-12.1.1-5825-linux-x64-standard.rpm
sudo dpkg –i adobe-substance-3d-designer_12.1.1-2_amd64.deb

[Polar1s]

Thanks for testing the commands! Would you mind showing me the errors from sbsrender? We did test both tools on Ubuntu, Windows, and MacOS, so I don't believe this is a prevalent issue.

Your method of installing SD on Ubuntu looks alright. The problem's nature will depend on what errors you get from sbsrender, but licensing seems unlikely since these two command-line executables don't perform such checking explicitly.

It's already late in my local time zone, so I will get back to you tomorrow once you have more results.

[yuyingyeh]

Thanks for testing the commands! Would you mind showing me the errors from sbsrender? We did test both tools on Ubuntu, Windows, and MacOS, so I don't believe this is a prevalent issue.

Your method of installing SD on Ubuntu looks alright. The problem's nature will depend on what errors you get from sbsrender, but licensing seems unlikely since these two command-line executables don't perform such checking explicitly.

It's already late in my local time zone, so I will get back to you tomorrow once you have more results.

I really appreciate your fast response! For the output from sbsrender, it is empty for the outputs

[
    {
        "identifier": "concrete_raw_panels",
        "outputs": [
        ]
    }
]

[Polar1s]

Below is the output from sbsrender when everything behaves as expected.

[
    {
        "identifier": "concrete_raw_panels",
        "outputs": [
            {
                "identifier": "bnw_spots_3_0_bnw_spots_3",
                "label": "bnw_spots_3_0_bnw_spots_3",
                "type": "image",
                "uid": 1933373088,
                "usages": [
                ],
                "value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/bnw_spots_3_0_bnw_spots_3.png"
            },
            {
                "identifier": "grunge_map_013_0_output",
                "label": "grunge_map_013_0_output",
                "type": "image",
                "uid": 1974574837,
                "usages": [
                ],
                "value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/grunge_map_013_0_output.png"
            },
            {
                "identifier": "clouds_3_0_clouds_3",
                "label": "clouds_3_0_clouds_3",
                "type": "image",
                "uid": 1938146574,
                "usages": [
                ],
                "value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/clouds_3_0_clouds_3.png"
            },
            {
                "identifier": "fractal_sum_base_0_fractal_sum_base",
                "label": "fractal_sum_base_0_fractal_sum_base",
                "type": "image",
                "uid": 1920358842,
                "usages": [
                ],
                "value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/fractal_sum_base_0_fractal_sum_base.png"
            },
            {
                "identifier": "fractal_sum_base_1_fractal_sum_base",
                "label": "fractal_sum_base_1_fractal_sum_base",
                "type": "image",
                "uid": 1924174953,
                "usages": [
                ],
                "value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/fractal_sum_base_1_fractal_sum_base.png"
            },
            {
                "identifier": "grunge_map_007_0_output",
                "label": "grunge_map_007_0_output",
                "type": "image",
                "uid": 2001691694,
                "usages": [
                ],
                "value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/grunge_map_007_0_output.png"
            },
            {
                "identifier": "tile_generator_0_tilegenerator",
                "label": "tile_generator_0_tilegenerator",
                "type": "image",
                "uid": 2009152657,
                "usages": [
                ],
                "value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/tile_generator_0_tilegenerator.png"
            }
        ]
    }
]

Since your outputs field is empty, it's possible that the generated concrete_raw_panels_input.sbs is incomplete or corrupted. My recommended next step is to open the SBS file and verify that you see seven pairs of generator and output nodes inside. Otherwise, something is wrong with exporting the input texture generation graph.

[yuyingyeh]

I use the same .sbs file to run below command on MacOS and Ubuntu:

<path/to/SD>/sbscooker concrete_raw_panels_input.sbs --output-path "."
<path/to/SD>/sbsrender render concrete_raw_panels_input.sbsar --output-format png --output-name "{outputNodeName}" --output-path "."

I can see those 7 files when I run on MacOS, but not for Ubuntu. I think my .sbs file is correct.

[Polar1s]

Thanks for the info. At this point, I tend not to think there is any fundamental issue with diffmat since the erratic behavior you reported is platform-specific and not widely reproducible. Therefore, I suggest taking the issue offline and relegating further correspondence to emails. Meanwhile, the case will stay open, and I'm happy to help with potential updates.

[yuyingyeh]

Hi @Polar1s ,

I have tried several versions of Substance Designer and found that version 11.1.2 works well on Ubuntu but the versions after 11.2.0 seem to have some issues. For version 11.1.2 , we need to set default toolkit root to /opt/Allegorithmic/Substance_Designer.

[Polar1s]

I appreciate your testing on previous SD versions. Release 11.1.2 was probably the last before Adobe rebranded SD as their product; thus, the default toolkit root was different from the latest version.

Since I have 11.2.2 installed on my Ubuntu system, I plan to install 12.1.1 using your method and see how the command-line tools work.

[Polar1s]

I am closing the issue for now. I concluded from my investigation that newer SD versions (after 11.1.2) might be incompatible with DiffMat in some systems, where the sbsrender program fails to generate output textures. The problem has not been proven widely reproducible and therefore does not affect DiffMat's usage.

However, we will improve DiffMat's documentation to address the earlier questions raised by @yuyingyeh. Thanks!