Design API for custom materials in a-frame component "io3d-data3d"

[frederic-schwarz]

End goal: Allow people to assign custom materials & textures to io3d-data3d entities. Components: io3d-data3d

After @tomas-polach asked an important question during #4, Madlaina brought up the question how we would do this for io3d-data3d.

The possibility to change textures and colors on materials is opening up many possibilities, e.g. changing the diffuse map without having to change the lightmap allows more dynamic materials without needing to re-bake.

The outcome should fulfill the following criteria:

• [ ] It should be possible to set & edit materials using the inspector
• [ ] It should be possible to set & edit materials using the HTML API (i.e. <a-entity base-data3d="" base-material="colorDiffuse: green">)
• [ ] (optional?) It should be possible to add a base-material to any a-entity to get our shaders onto any a-entity

[Kasparlogic]

This has relevance for Home Story as we would like to change the materials in a backed model

[Aadjou]

Following problems need to be solved to make material editability on baked models a good experience for users:

• baked models at the moment have a very unstructured material setup

  • due to limiting draw calls, all materials with the same properties get unified
  • For each lightmap- material combination, a new material is required
  • after lightmap distribution, tris with the same material could be split in different subgroups (mat0_lm0, mat0_lm1 ... etc)
  • this leads to the possibility of connected triangles getting divided into seperate materials -> really annoying for editability (because we loose the semantic information about floors, walls etc)

• Ideas on solving these problems: (i'm not sure if all these ideas could work out, since it's just a quick memo)

  • Tag materials/meshes that should stay intact in data3d (-> baking will know which objects it should not rip apart)
  • allow material properties override in data3d
  • The unique material is referenced once and the data3d meshes properties override the lightmap value
  • for changing materials in the data3dview - change material with identical properties (but ignore the lightmap textures as a property)

of course we can already offer editability before solving this issue. but for user experience I recommend that this should be adressed.

@Kasparlogic @frederic-schwarz @AVGP we should probably discuss this soon in a meeting

[Kasparlogic]

@Aadjou yes please, taking it to slack

[tomas-polach]

updated title to be more specific (considering that not all issues are related to a-frame components)

[tomas-polach]

btw. any updates on this one?

[urish]

I'd also love to have this, as the current workflow really slows down prototyping, having to wait for the bake process to complete whenever I make even the slightest change to the material configuration. Please make this happen 👍

[tomas-polach]

@Aadjou thx for pointing this out. another solution proposals:

• being explicit about baking materials by allowing meshes to have material and lightMapMaterial properties. this would be backwards compatible (with old models) and to get an unbaked version one could just delete the lightMapMaterial property from meshes.

regarding the material mixup after baking: i suggest to introduce a "Do not optimize model" checkbox in lighting tab. this would result in optimizing function not being used, keeping the original material structure in place. doing this on a material level in the editor would be quite exhausting because we do not have a multiselect feature.

[tomas-polach]

API design proposal for embedded material definitions:

<style type="text/data3d">
#house .wall27, .wall12 {
  diffuse-color: #0ee;
  diffuse-texture: url('https://...');
  specular-texture: storageId('/whatever...');
}
</style>

<scene>
  <a-entity id="house" io3d-data3d="url:http://..."></a-entity>
</scene>

@AVGP if i am not mistaking, style elements with custom type will not be parsed as CSS by the browser. (it works like that with the script element. scripts of custom type will be simply ignored). will jsfiddle it in a sec ...

https://jsfiddle.net/3dio/spv10272/ chrome, firefox and safari do not complain at all :) can anybody try IE pls?

[tomas-polach]

using CSS syntax would be cool:

• it's familiar to web devs
• get highlighted correctly in IDEs
• suitable for many materials (instead of squeezing them into attributes)
• can be loaded from separate file using <link rel="stylesheet" type="text/data3d" href="mystyle.css">

[tomas-polach]

@urish what do you think?

[urish]

I'm not sure CSS would be the best way to go, see the following discussion:

https://github.com/aframevr/aframe/issues/1102

[tomas-polach]

@urish thx for your feedback.

to unblock you i wrote some example code using undocumented data3dView methods: https://codepen.io/tomas-polach/pen/rGVjbg?editors=1010

pls make sure to use the specific io3d.js version from the example (not the "1.x.x" dist URL) because the data3dView methods will most likely change in the future.

does that help you or did i miss something?

[urish]

Looks great, thank you!

[tomas-polach]

happy to help! would love to know what you are working on (...if it's not confidential)

[urish]

Sure, it is a VR Workshop for AngularConnect

[tomas-polach]

that's cool!!

this might be interesting for you as well: https://github.com/archilogic-com/aframe-gblock

allows to load google blocks models directly as components (material changes there work differently because its indexed buffergeometries with vertex colors) but you can get a lot of 3d content and build a cool scene pretty quickly

[urish]

Awesome, thanks! I love that :)

[tomas-polach]

proposal using a-asset-item as proposed by @frederic-schwarz in #94

support for custom materials could look like this:

  <a-assets>
     <a-asset-item id="my-mat" io3d-material="colorDiffuse: #fff; colorSpecular: #09f; mapDiffuse: texture1.jps"></a-asset-item>
  </a-assets>
<a-entity io3d-wall="h:2.4, l:3, material_front: #my-mat" position="2 0 -1" rotation="0 12 0"></a-entity>

[tomas-polach]

this is also relevant for the architecture toolkit: https://github.com/archilogic-com/3dio-js/blob/0191bf01071d07477d40b3fc631dc118038b7e23/src/scene/structure/to-aframe-elements.js#L373