Closed kwokcb closed 5 months ago
jstone-lucasfilm
commented
1 year ago This is a great topic for discussion, @kwokcb, and thanks for getting it started!
It may be user error on my own part, but when I compare the character count of the two representations of your example in Sublime Text I get 3581 characters for XML and 5062 characters for JSON, which seems comparable to the results we've seen in earlier tests.
Am I miscounting or misunderstanding what I should expect?
kwokcb
commented
1 year ago Hi @jstone-lucasfilm, The examples are the default formatting so yes they are larger. It's all the whitespace padding added. There are a few possible options available:
Or a custom string serializer which handles each JSON element. I haven't tried this but would mean you don't really need to use a external library to produce JSON (nlohmann json was tested).
jstone-lucasfilm
commented
1 year ago @kwokcb I think you're right to use default formatting here, and I'm not implying that we should additionally compress either XML or JSON files by removing whitespace or switching to tabs, but it's worth noting that the JSON files are larger than their XML equivalents. This isn't a deal breaker for the idea of supporting JSON as an alternative format, but it matches the results that we saw in earlier tests of this idea.
kwokcb
commented
1 year ago I think even for readability it would be good to make it more compact (as with XML) so you JSON you get something like:
"add:sum": {
"input:in1": { "nodename": "scale_xyz", "type": "float" },
"input:in2": { "nodename": "scale_noise", "type": "float" },
"type": "float"
} <add name="sum" type="float">
<input name="in1" type="float" nodename="scale_xyz" />
<input name="in2" type="float" nodename="scale_noise" />
</add>Looking at the three.js prototype implementation which consumes XML currently:
MaterialXLoader) parses MTLX files using an XML serializer, and creates a node dictionary (for lookup to perform links for instance). jstone-lucasfilm
commented
5 months ago From the discussion above, it sounds like we should continue to focus on MTLX files within the MaterialX project, with the expectation that external projects will make different choices for serialization and deserialization of MaterialX content (e.g. USDA, JSON).
I'll go ahead and close out this issue for now, but feel free to bring this discussion back if needed in the future.
fire
commented
5 months ago For reference this was implemented in https://github.com/kwokcb/materialxjson
Proposal
This is an initial "strawman" proposal for including JSON serialization support as part of the core of MaterialX. A suitable library would probably be
MaterialXFormatwhere XML serialization exists.Reasons for JSON vs XML
Implementation Notes
Base Requirements
Examples:
This are some examples with possible format (based on prototyping)
Marble Example (XML)
```xmlMarble Example (JSON)
glTF "Olives" Sample Library (XML)
```xmlThis shows a potential target for integration which would be inclusion as part of a GLTF document
glTF "Olives" Sample Library (GLTF)
```JSON { "asset": { "copyright": "Copyright 2022-2023: Bernard Kwok", "generator": "glTF_MaterialX generator: MaterialX 1.38.8 to glTF 2.0 .", "version": "2.0" }, "images": [ { "name": "image_basecolor2", "uri": "goldleaf_col.png" }, { "name": "image_orm3", "uri": "goldleaf_orm.png" }, { "name": "image_normal3", "uri": "goldleaf_nrm.png" }, { "name": "image_basecolor", "uri": "olives_col.png" }, { "name": "image_orm", "uri": "olives_orm.png" }, { "name": "image_normal", "uri": "olives_nrm.png" }, { "name": "image_occlusion", "uri": "goldleaf_orm.png" }, { "name": "image_iridescence", "uri": "glassdish_irid.png" }, { "name": "image_iridescence_thickness", "uri": "glassdish_irid.png" }, { "name": "image_orm2", "uri": "glasscover_orm.png" }, { "name": "image_normal2", "uri": "glasscover_nrm.png" }, { "name": "image_iridescence2", "uri": "glasscover_irid.png" }, { "name": "image_iridescence_thickness2", "uri": "glasscover_irid.png" } ], "materials": [ { "name": "SHD_goldLeaf", "pbrMetallicRoughness": { "baseColorTexture": { "index": 0 }, "metallicRoughnessTexture": { "index": 1 } }, "extensions": { "KHR_materials_clearcoat": { }, "KHR_materials_ior": { }, "KHR_materials_specular": { }, "KHR_materials_transmission": { }, "KHR_materials_sheen": { }, "KHR_materials_emissive_strength": { }, "KHR_materials_iridescence": { } }, "occlusionTexture": { "index": 1, "scale": 0 }, "alphaMode": "MASK" }, { "name": "SHD_olives", "pbrMetallicRoughness": { "baseColorTexture": { "index": 3 }, "metallicRoughnessTexture": { "index": 4 } }, "extensions": { "KHR_materials_clearcoat": { }, "KHR_materials_ior": { }, "KHR_materials_specular": { }, "KHR_materials_transmission": { }, "KHR_materials_sheen": { }, "KHR_materials_emissive_strength": { }, "KHR_materials_iridescence": { } }, "occlusionTexture": { "index": 4, "scale": 0 } }, { "name": "SHD_glassDish", "pbrMetallicRoughness": { "metallicFactor": 0, "roughnessFactor": 0.0700000003 }, "extensions": { "KHR_materials_clearcoat": { }, "KHR_materials_ior": { }, "KHR_materials_specular": { }, "KHR_materials_transmission": { "transmissionFactor": 1 }, "KHR_materials_sheen": { }, "KHR_materials_emissive_strength": { }, "KHR_materials_iridescence": { "iridescenceFactor": 1, "iridescenceTexture": { "index": 7, "scale": 0 }, "iridescenceThicknessMinimum": 500, "iridescenceThicknessMaximum": 550, "iridescenceThicknessTexture": { "index": 8 } } }, "occlusionTexture": { "index": 6, "scale": 0 } }, { "name": "SHD_glassCover", "pbrMetallicRoughness": { "metallicRoughnessTexture": { "index": 9 } }, "extensions": { "KHR_materials_clearcoat": { }, "KHR_materials_ior": { }, "KHR_materials_specular": { }, "KHR_materials_transmission": { "transmissionFactor": 1 }, "KHR_materials_sheen": { }, "KHR_materials_emissive_strength": { }, "KHR_materials_iridescence": { "iridescenceFactor": 1, "iridescenceTexture": { "index": 11, "scale": 0 }, "iridescenceThicknessMinimum": 500, "iridescenceThicknessMaximum": 550, "iridescenceThicknessTexture": { "index": 12 } } }, "occlusionTexture": { "index": 9, "scale": 0 } } ], "textures": [ { "name": "image_basecolor2", "source": 0 }, { "name": "image_orm3", "source": 1 }, { "name": "image_normal3", "source": 2 }, { "name": "image_basecolor", "source": 3 }, { "name": "image_orm", "source": 4 }, { "name": "image_normal", "source": 5 }, { "name": "image_occlusion", "source": 6 }, { "name": "image_iridescence", "source": 7 }, { "name": "image_iridescence_thickness", "source": 8 }, { "name": "image_orm2", "source": 9 }, { "name": "image_normal2", "source": 10 }, { "name": "image_iridescence2", "source": 11 }, { "name": "image_iridescence_thickness2", "source": 12 } ], "extensionsUsed": [ "KHR_materials_clearcoat", "KHR_materials_ior", "KHR_materials_specular", "KHR_materials_transmission", "KHR_materials_sheen", "KHR_materials_emissive_strength", "KHR_materials_iridescence" ] } ```glTF "Olives" Sample Library (JSON)
```JSON { "materialx": { "colorspace": "lin_rec709", "nodegraph:NG_marble1": { "add:bias": { "input:in1": { "nodename": "scale", "type": "float" }, "input:in2": { "type": "float", "value": "0.5" }, "type": "float" }, "add:sum": { "input:in1": { "nodename": "scale_xyz", "type": "float" }, "input:in2": { "nodename": "scale_noise", "type": "float" }, "type": "float" }, "dotproduct:add_xyz": { "input:in1": { "nodename": "obj_pos", "type": "vector3" }, "input:in2": { "type": "vector3", "value": "1, 1, 1" }, "type": "float" }, "fractal3d:noise": { "input:octaves": { "interfacename": "noise_octaves", "type": "integer" }, "input:position": { "nodename": "scale_pos", "type": "vector3" }, "type": "float" }, "input:base_color_1": { "type": "color3", "uifolder": "Marble Color", "uiname": "Color 1", "value": "0.8, 0.8, 0.8" }, "input:base_color_2": { "type": "color3", "uifolder": "Marble Color", "uiname": "Color 2", "value": "0.1, 0.1, 0.3" }, "input:noise_octaves": { "type": "integer", "uifolder": "Marble Noise", "uiname": "Octaves", "uisoftmax": "8", "uisoftmin": "1", "value": "3" }, "input:noise_power": { "type": "float", "uifolder": "Marble Noise", "uiname": "Power", "uisoftmax": "10.0", "uisoftmin": "1.0", "value": "3.0" }, "input:noise_scale_1": { "type": "float", "uifolder": "Marble Noise", "uiname": "Scale 1", "uisoftmax": "10.0", "uisoftmin": "1.0", "value": "6.0" }, "input:noise_scale_2": { "type": "float", "uifolder": "Marble Noise", "uiname": "Scale 2", "uisoftmax": "10.0", "uisoftmin": "1.0", "value": "4.0" }, "mix:color_mix": { "input:bg": { "interfacename": "base_color_1", "type": "color3" }, "input:fg": { "interfacename": "base_color_2", "type": "color3" }, "input:mix": { "nodename": "power", "type": "float" }, "type": "color3" }, "multiply:scale": { "input:in1": { "nodename": "sin", "type": "float" }, "input:in2": { "type": "float", "value": "0.5" }, "type": "float" }, "multiply:scale_noise": { "input:in1": { "nodename": "noise", "type": "float" }, "input:in2": { "type": "float", "value": "3.0" }, "type": "float" }, "multiply:scale_pos": { "input:in1": { "nodename": "obj_pos", "type": "vector3" }, "input:in2": { "interfacename": "noise_scale_2", "type": "float" }, "type": "vector3" }, "multiply:scale_xyz": { "input:in1": { "nodename": "add_xyz", "type": "float" }, "input:in2": { "interfacename": "noise_scale_1", "type": "float" }, "type": "float" }, "output:out": { "nodename": "color_mix", "type": "color3" }, "position:obj_pos": { "type": "vector3" }, "power:power": { "input:in1": { "nodename": "bias", "type": "float" }, "input:in2": { "interfacename": "noise_power", "type": "float" }, "type": "float" }, "sin:sin": { "input:in": { "nodename": "sum", "type": "float" }, "type": "float" } }, "standard_surface:SR_marble1": { "input:base": { "type": "float", "value": "1" }, "input:base_color": { "nodegraph": "NG_marble1", "output": "out", "type": "color3" }, "input:specular_roughness": { "type": "float", "value": "0.1" }, "input:subsurface": { "type": "float", "value": "0.4" }, "input:subsurface_color": { "nodegraph": "NG_marble1", "output": "out", "type": "color3" }, "type": "surfaceshader", "xpos": "13.768116", "ypos": "-0.672414" }, "surfacematerial:Marble_3D": { "input:surfaceshader": { "nodename": "SR_marble1", "type": "surfaceshader" }, "type": "material", "xpos": "17.391304", "ypos": "0.000000" }, "version": "1.38" }, "mimetype": "application/mtlx+json" } ```