Closed yuyingyeh closed 2 years ago
Hi Yu-Ying,
This error happens when DiffMat tries to translate a generator node (e.g., bnw_spots_3) and fails. This is expected since DiffMat doesn't implement any generator node in the current version. To circumvent this error, add an -e
option to the test script to generate input textures using Substance Designer command-line tools.
We noticed that neither README.md nor the source file adequately referenced this error. We will make necessary improvements to documentation in the next release. For now, please regard the -e
option as mandatory.
Thanks!
Hi Beichen,
Thanks for your reply! I have tried the same command with -e
option:
python3 test_sampler.py sbs/match_v1/concrete_raw_panels.sbs -e -t /opt/Adobe/Adobe_Substance_3D_Designer -r test_diffmat
but encounter this error
Traceback (most recent call last):
File "test_sampler.py", line 99, in <module>
main()
File "test_sampler.py", line 95, in main
img_format=img_format)
File "/usr/local/lib/python3.7/dist-packages/diffmat/optim/sampler.py", line 229, in evaluate
maps = graph.evaluate_maps()
File "/usr/local/lib/python3.7/dist-packages/diffmat/core/graph.py", line 156, in evaluate_maps
memory.update({key: val for key, val in zip(inst['result'], result) if key})
TypeError: zip argument #2 must support iteration
Unfortunately, I wasn't able to reproduce the error on my Ubuntu 18.04 (SD version 11.2.2) or Windows 10 (SD version 12.1.1) system. Could you tell me the Substance Designer version you're using?
In addition, identifying which node triggered the error also helps. You could add a -l verbose
option to enable full-scale logging, where diffmat will time each node execution. The problematic node is after the last successful timing entry.
I'm using version 12.1.1 on Ubuntu 18.04. Actually here is the node info coming out before the error message:
[INFO] Compiled material graph program (51 nodes):
bnw_spots_3_0: () -> bnw_spots_3_0_bnw_spots_3
uniform_0: () -> uniform_0
grunge_map_013_0: () -> grunge_map_013_0_output
uniform_1: () -> uniform_1
clouds_3_0: () -> clouds_3_0_clouds_3
fractal_sum_base_0: () -> fractal_sum_base_0_fractal_sum_base
uniform_2: () -> uniform_2
fractal_sum_base_1: () -> fractal_sum_base_1_fractal_sum_base
grunge_map_007_0: () -> grunge_map_007_0_output
tile_generator_0: () -> tile_generator_0_tilegenerator
gradient_0: (bnw_spots_3_0_bnw_spots_3) -> gradient_0
curvature_smooth_0: (uniform_1) -> curvature_smooth_0
blur_2: (clouds_3_0_clouds_3) -> blur_2
levels_7: (clouds_3_0_clouds_3) -> levels_7
blur_0: (fractal_sum_base_0_fractal_sum_base) -> blur_0
slope_blur_grayscale_2: (fractal_sum_base_0_fractal_sum_base, clouds_3_0_clouds_3) -> slope_blur_grayscale_2
levels_5: (fractal_sum_base_1_fractal_sum_base) -> levels_5
safe_transform_grayscale_0: (grunge_map_007_0_output) -> safe_transform_grayscale_0
blur_1: (tile_generator_0_tilegenerator) -> blur_1
slope_blur_0: (gradient_0, grunge_map_007_0_output) -> slope_blur_0
levels_1: (curvature_smooth_0) -> levels_1
levels_3: (curvature_smooth_0) -> levels_3
blend_0: (bnw_spots_3_0_bnw_spots_3, levels_7, levels_5) -> blend_0
safe_transform_grayscale_1: (levels_5) -> safe_transform_grayscale_1
blend_1: (bnw_spots_3_0_bnw_spots_3, safe_transform_grayscale_0, None) -> blend_1
slope_blur_grayscale_3: (blur_1, blur_1) -> slope_blur_grayscale_3
levels_4: (slope_blur_0) -> levels_4
blend_4: (uniform_2, uniform_2, levels_3) -> blend_4
blend_12: (levels_3, uniform_0, None) -> blend_12
blend_10: (slope_blur_grayscale_2, blend_0, safe_transform_grayscale_1) -> blend_10
histogram_scan_0: (blend_1) -> histogram_scan_0
levels_2: (slope_blur_grayscale_3) -> levels_2
blend_8: (grunge_map_013_0_output, blend_10, None) -> blend_8
blend_7: (histogram_scan_0, levels_1, None) -> blend_7
slope_blur_grayscale_1: (levels_2, blur_2) -> slope_blur_grayscale_1
blend_2: (grunge_map_007_0_output, blend_8, None) -> blend_2
blend_5: (levels_4, blend_4, blend_7) -> blend_5
blend_9: (blend_7, blend_12, None) -> blend_9
slope_blur_grayscale_0: (slope_blur_grayscale_1, blur_0) -> slope_blur_grayscale_0
hsl_0: (blend_5) -> hsl_0
sharpen_0: (blend_9) -> sharpen_0
levels_6: (slope_blur_grayscale_0) -> levels_6
levels_0: (hsl_0) -> levels_0
invert_grayscale_0: (levels_6) -> invert_grayscale_0
basecolor: (levels_0) -> basecolor
blend_3: (bnw_spots_3_0_bnw_spots_3, invert_grayscale_0, None) -> blend_3
blend_6: (blend_3, slope_blur_grayscale_1, None) -> blend_6
blend_11: (blend_6, blend_2, None) -> blend_11
normal_0: (blend_11) -> normal_0
normal: (normal_0) -> normal
roughness: (sharpen_0) -> roughness
[INFO] Graph name: concrete_raw_panels #nodes: 51 #params: 2138
I have tried -l verbose
but nothing changed. It seems that the format of result has some issues here...?
https://github.com/mit-gfx/diffmat/blob/bd7a8d5d0e500e6b583af30697e82a518103e522/diffmat/core/graph.py#L156
I have elevated this issue to a known bug in the system.
If no [DEBUG] message follows the screen output above, my best guess is that input textures were not correctly generated or read, so the result
of the first node is None
(hence not iterable). What you can do is check the test_diffmat/concrete_raw_panels/external_input/default
directory for generated input textures. The error should have come from reading if the following files are present.
Also, does the problem occur to other graphs (*.sbs files) you have tested?
I have elevated this issue to a known bug in the system.
If no [DEBUG] message follows the screen output above, my best guess is that input textures were not correctly generated or read, so the
result
of the first node isNone
(hence not iterable). What you can do is check thetest_diffmat/concrete_raw_panels/external_input/default
directory for generated input textures. The error should have come from reading if the following files are present.
I only found concrete_raw_panels_input.sbs
in test_diffmat/concrete_raw_panels/external_input/default
. It seems that the input textures are not generated in my case.
Also, does the problem occur to other graphs (*.sbs files) you have tested?
I have tried aluminium_cut.sbs
, but the same issue happened.
I have elevated this issue to a known bug in the system. If no [DEBUG] message follows the screen output above, my best guess is that input textures were not correctly generated or read, so the
result
of the first node isNone
(hence not iterable). What you can do is check thetest_diffmat/concrete_raw_panels/external_input/default
directory for generated input textures. The error should have come from reading if the following files are present.I only found
concrete_raw_panels_input.sbs
intest_diffmat/concrete_raw_panels/external_input/default
. It seems that the input textures are not generated in my case.
For aluminium_cut.sbs
, there is a single line [DEBUG]
message after [INFO]
message:
[INFO] (diffmat.core - sampler.py) Graph name: aluminium_cut #nodes: 29 #params: 46
[DEBUG] (diffmat.core - base.py) Node uniform_0: 2.852 ms
It makes sense that the issue repeats on other graphs if it is related to the input texture generator itself.
Did you see any warning or error messages from the SD executables? Diffmat typically invokes SD command-line tools (sbscooker and sbsrender) to generate input textures in the external_input/default
folder. In your case, it seems unusual to produce neither input textures nor warning/error messages.
Nonetheless, you could try manually generating the textures. Enter the test_diffmat/concrete_raw_panels/external_input/default
folder and run the following shell command:
/opt/Adobe/Adobe_Substance_3D_Designer/sbscooker concrete_raw_panels_input.sbs --output-path "."
/opt/Adobe/Adobe_Substance_3D_Designer/sbsrender render concrete_raw_panels_input.sbsar --output-format png --output-name "{outputNodeName}" --output-path "."
Let me know if these commands generate the textures without warning/error messages.
It makes sense that the issue repeats on other graphs if it is related to the input texture generator itself.
Did you see any warning or error messages from the SD executables? Diffmat typically invokes SD command-line tools (sbscooker and sbsrender) to generate input textures in the
external_input/default
folder. In your case, it seems unusual to produce neither input textures nor warning/error messages.Nonetheless, you could try manually generating the textures. Enter the
test_diffmat/concrete_raw_panels/external_input/default
folder and run the following shell command:/opt/Adobe/Adobe_Substance_3D_Designer/sbscooker concrete_raw_panels_input.sbs --output-path "." /opt/Adobe/Adobe_Substance_3D_Designer/sbsrender render concrete_raw_panels_input.sbsar --output-format png --output-name "{outputNodeName}" --output-path "."
Let me know if these commands generate the textures without warning/error messages.
I have tried the two commands. The first one works but not for the second one with version 11.2.2 and 12.1.1 on Ubuntu. Both commands work with version 12.1.1 on MacOS.
Since Adobe only release .rpm version for Linux, I need to convert it to .deb with alien. I'm not sure if something went wrong during this conversion or any license information required during the installation. Basically I installed the SD with commands:
sudo alien Adobe_Substance_3D_Designer-12.1.1-5825-linux-x64-standard.rpm
sudo dpkg –i adobe-substance-3d-designer_12.1.1-2_amd64.deb
Thanks for testing the commands! Would you mind showing me the errors from sbsrender? We did test both tools on Ubuntu, Windows, and MacOS, so I don't believe this is a prevalent issue.
Your method of installing SD on Ubuntu looks alright. The problem's nature will depend on what errors you get from sbsrender, but licensing seems unlikely since these two command-line executables don't perform such checking explicitly.
It's already late in my local time zone, so I will get back to you tomorrow once you have more results.
Thanks for testing the commands! Would you mind showing me the errors from sbsrender? We did test both tools on Ubuntu, Windows, and MacOS, so I don't believe this is a prevalent issue.
Your method of installing SD on Ubuntu looks alright. The problem's nature will depend on what errors you get from sbsrender, but licensing seems unlikely since these two command-line executables don't perform such checking explicitly.
It's already late in my local time zone, so I will get back to you tomorrow once you have more results.
I really appreciate your fast response! For the output from sbsrender, it is empty for the outputs
[
{
"identifier": "concrete_raw_panels",
"outputs": [
]
}
]
Below is the output from sbsrender when everything behaves as expected.
[
{
"identifier": "concrete_raw_panels",
"outputs": [
{
"identifier": "bnw_spots_3_0_bnw_spots_3",
"label": "bnw_spots_3_0_bnw_spots_3",
"type": "image",
"uid": 1933373088,
"usages": [
],
"value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/bnw_spots_3_0_bnw_spots_3.png"
},
{
"identifier": "grunge_map_013_0_output",
"label": "grunge_map_013_0_output",
"type": "image",
"uid": 1974574837,
"usages": [
],
"value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/grunge_map_013_0_output.png"
},
{
"identifier": "clouds_3_0_clouds_3",
"label": "clouds_3_0_clouds_3",
"type": "image",
"uid": 1938146574,
"usages": [
],
"value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/clouds_3_0_clouds_3.png"
},
{
"identifier": "fractal_sum_base_0_fractal_sum_base",
"label": "fractal_sum_base_0_fractal_sum_base",
"type": "image",
"uid": 1920358842,
"usages": [
],
"value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/fractal_sum_base_0_fractal_sum_base.png"
},
{
"identifier": "fractal_sum_base_1_fractal_sum_base",
"label": "fractal_sum_base_1_fractal_sum_base",
"type": "image",
"uid": 1924174953,
"usages": [
],
"value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/fractal_sum_base_1_fractal_sum_base.png"
},
{
"identifier": "grunge_map_007_0_output",
"label": "grunge_map_007_0_output",
"type": "image",
"uid": 2001691694,
"usages": [
],
"value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/grunge_map_007_0_output.png"
},
{
"identifier": "tile_generator_0_tilegenerator",
"label": "tile_generator_0_tilegenerator",
"type": "image",
"uid": 2009152657,
"usages": [
],
"value": "D:/diffmat_release/test/test_diffmat/concrete_raw_panels/external_input/default/tile_generator_0_tilegenerator.png"
}
]
}
]
Since your outputs
field is empty, it's possible that the generated concrete_raw_panels_input.sbs
is incomplete or corrupted. My recommended next step is to open the SBS file and verify that you see seven pairs of generator and output nodes inside. Otherwise, something is wrong with exporting the input texture generation graph.
I use the same .sbs
file to run below command on MacOS and Ubuntu:
<path/to/SD>/sbscooker concrete_raw_panels_input.sbs --output-path "."
<path/to/SD>/sbsrender render concrete_raw_panels_input.sbsar --output-format png --output-name "{outputNodeName}" --output-path "."
I can see those 7 files when I run on MacOS, but not for Ubuntu. I think my .sbs
file is correct.
Thanks for the info. At this point, I tend not to think there is any fundamental issue with diffmat since the erratic behavior you reported is platform-specific and not widely reproducible. Therefore, I suggest taking the issue offline and relegating further correspondence to emails. Meanwhile, the case will stay open, and I'm happy to help with potential updates.
Hi @Polar1s ,
I have tried several versions of Substance Designer and found that version 11.1.2
works well on Ubuntu but the versions after 11.2.0
seem to have some issues. For version 11.1.2
, we need to set default toolkit root to /opt/Allegorithmic/Substance_Designer
.
I appreciate your testing on previous SD versions. Release 11.1.2
was probably the last before Adobe rebranded SD as their product; thus, the default toolkit root was different from the latest version.
Since I have 11.2.2
installed on my Ubuntu system, I plan to install 12.1.1
using your method and see how the command-line tools work.
I am closing the issue for now. I concluded from my investigation that newer SD versions (after 11.1.2) might be incompatible with DiffMat in some systems, where the sbsrender program fails to generate output textures. The problem has not been proven widely reproducible and therefore does not affect DiffMat's usage.
However, we will improve DiffMat's documentation to address the earlier questions raised by @yuyingyeh. Thanks!
As title, I'd like to run the test script. It shows that the configuration file for node type: bnw_spots_3 for graph concrete_raw_panels.sbs is missing. Any helps will be appreciated! Thanks!