glTF 2.0 loader

[andreasplesch]

An issue to collect infos and ideas for a glTF v. 2.0 loader to be used by externalGeometry/externalShape and potentially Inline

https://github.com/KhronosGroup/glTF/pull/826 is the best starting point to find out about changes from v. 1.0.1

https://github.com/mrdoob/three.js/issues/10806 also is a good summary and starting point.

v. 2.0 spec. ETA ? This spring/summer ?

lots of breaking changes in the json structure

• binary glTF slightly different, readHeader/Scene/Body should be straightforward to adjust
• use of arrays requires reworking
• check if v.1.0 loader uses name property value or key name itself.

Strategy:

start from scratch (but with v.1 loader at hand) or tweak v.1.0 loader ? probably tweak new gltf to x3d mapping ?: externalShape should be named primitive + its material (without # just first mesh or like now?) externalGeometry should be primitive geometry externalGroup with .glb#nodename should be node transform + children + mesh (primitives) externalGroup should be scene with all nodes, no lights, no cameras externalLight ? externalViewpoint ? Inline should be Inline as much as possible with namespacename

First target just externalShape based on existing implementation.

[andreasplesch]

PBR paper:

http://www.instantuv.org/wp-content/uploads/2017/03/pbr2016.pdf

[andreasplesch]

Here is a glitch which I use to explore glTF 2.0 x3dom loading: https://x3dom-gltf2-scratch.glitch.me/ https://glitch.com/edit/#!/x3dom-gltf2-scratch?path=glTFLoader.js:27:0 The duck is glTF2.0, the truck is glTF1.0.

@tsturm : The idea is to branch into glTF 2.0 version parsing if the version is detected and then use completely separated methods for parsing. These are in glTFLoader2.js . It turns out that minor adjustments to the glTF 1.0 parsing are sufficient for the mesh loading. PBR materials would be next but then requires a new shader, dealing with lights.

[tsturm]

Hi @andreasplesch,

looks good for me. I'm not sure how many people are using glTF 1.0 anymore, maybe we can scratch the support for it completely.

did you see my "gltf2" branch?

I have modified the Inline-Node, so that it's now possible to load glTF, glTF-Embedded & glTF-Binary simply per Inline like a normal X3D-File.

The branch is still WIP but you can find a first test here: https://examples.x3dom.org/x3dom-inline-gltf-2/

[andreasplesch]

This is great ! Yes, I saw the glTF2 branch and want to take a closer look. Is this compatible with externalGeometry/Shape ? Perhaps it is possible to use the PBR shader with externalShape ? I agree that glTF2 is replacing glTF1, and Inline would not need to support glTF1. If possible it would be nice to keep existing glTF1 loading available. As an aside, I noticed that on Chrome Android the textures on the Duck and Milk truck are off, not using the correct uvs. On Chrome Windows they are fine.

[tsturm]

No, it doesn't use the ExternalShape/Geometry-Nodes for rendering. I have introduced a new Geometry-Node called BufferGeometry, which makes it possible to access binary data simply in a declarative way.

The rendering is fully integrated in x3dom and goes through the normal x3dom-rendering-pipeline. There is no need for special glTF code to handle texture loading, lights, volumes, picking, etc. anymore & we can access the Nodes per DOM like in a normal inlined X3D-Scene.

The PBR shader is one of the next steps. The plan is to integrate the PBR-Stuff directly in the default x3dom DynamicShader so every X3D Geometry-Node can use PBR in the future.

Thanks, for the hint that the UV's are broken on mobile devices. I will take a look.

[npolys]

HOORAY!!

you guys rock :-)

On Sun, Mar 11, 2018 at 10:21 AM, Timo Sturm notifications@github.com wrote:

No, it doesn't use the ExternalShape/Geometry-Nodes for rendering. I have introduced a new Geometry-Node called BufferGeometry, which makes it possible to access binary data simply in a declarative way.

The rendering is fully integrated in x3dom and goes through the normal x3dom-rendering-pipeline. There is no need for special glTF code to handle texture loading, lights, volumes, etc. anymore & we can access the Nodes per DOM like in a normal inlined X3D-Scene.

The PBR shader is one of the next steps. The plan is to integrate the PBR-Stuff directly in the default x3dom DynamicShader so every X3D Geometry-Node can use PBR in the future.

Thanks, for the hint that the UV's are broken on mobile devices. I will take a look.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/x3dom/x3dom/issues/724#issuecomment-372119056, or mute the thread https://github.com/notifications/unsubscribe-auth/AHU5M72vhKnKCrlesID-8ZEpY1H6N6Lyks5tdTLMgaJpZM4MMkMf .

-- Nicholas F. Polys, Ph.D.

Director of Visual Computing Virginia Tech Research Computing

Affiliate Professor Virginia Tech Department of Computer Science

[cx20]

X3DOM glTF 2.0 support is a great job.

I added X3DOM to gltf-test. https://github.com/cx20/gltf-test https://cx20.github.io/gltf-test/index.html

If support for PBR is added, I think the status will be updated.

[tsturm]

@cx20 Thanks! @andreasplesch UV's on mobile devices are fixed ;)

[andreasplesch]

ok, I see how glTF2Loader parses the glTF into DOM nodes. Some notes in no particular order:

• Material will need maps: Metalroughnesstexture, Opacity, Occlusion, Normal; overlap with commonsurfaceshader ?
• @mlimper has proposal for PBRMaterial
• gltf camera to x3d viewpoint may be easy to add
• https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL instead of dataurls perhaps better for embedded images
• add glTF names as DEF names ?

[tsturm]

@andreasplesch Thanks, for your fast review!

• I will add a special PBRMaterial-Node like the one I proposed in my Web3D-Paper 2016. It was also the base for the current glTF 2.0 spec.
• Yes, adding glTF Cameras & Animations as X3D-Viewpoints & ROUTES should be possible
• That's right ObjectURL's are maybe better.
• x3dom makes no difference between DEF's and ID's

[andreasplesch]

Apologies for the mixup, that was your proposal (see link above). BTW, https://github.com/castle-engine/view3dscene also has plans to add glTF2 support to Inlines, as it already can inline many other formats.

Animations: yes, it should be possible to translate glTF animations into TimeSensor/Interpolator/Routes, great to start on this. There was a discussion on glTF issues on semantics, eg. a way to specify when and how defined animations should be played: https://github.com/KhronosGroup/glTF/pull/1137

DEF's may be useful for X3D consistency. For example, it may make sense to serialize out the x3d DOM derived from the glTF. At some point, we may have a generic glTF2 to X3D translator, if we can get PBR into X3D.

weights and joints may be the final frontier.

[andreasplesch]

Just to confirm, UV's on android work for me as well now.

[tsturm]

No Problem, Max and I worked together on the Paper.

I also see the discussions on the glTF repo. I think we can start with a first basic implementation which starts the animations immediately & extend it later.

You are right, DEF's are more useful for consistency. I will change the ID's to DEF's.

Yes, weights and joints are definitely the final frontier.

[paulmasson]

FWIW examples look fine for iOS 11

[andreasplesch]

I added an input line for loading gltf urls: https://rawgit.com/andreasplesch/x3dom/687bd9abe5e07b75286dcca7baed0845f41fd418/test/functional/inline-gltf.html Avocado does not have buffergeometryview nodes https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Models/master/2.0/Avocado Any quick guesses why ? Could be an easier issue to work with to get more familiar, for somebody or me. Another one is to convert to ObjectURLs since they are designed to work with typedArrays/BufferViews: https://developer.mozilla.org/en-US/docs/Web/API/Blob ObjectURLs work with XHR. There may be a limit on how large dataurls can get ?

[tsturm]

There are still many glTF related special cases which are not handled right now. As mentioned it's an experimental WIP branch.

I've updated the example to the newest version and the Avocado should work now: https://examples.x3dom.org/x3dom-inline-gltf-2/

[tsturm]

ObjectURLs are also in now: https://examples.x3dom.org/x3dom-inline-gltf-2/

[andreasplesch]

Wow, that was quick. BarramundiFish works as well ;)

and Brainstem and Monster sans animation.

[cx20]

I confirmed that some models can be displayed, I updated the status of gltf-test. https://github.com/cx20/gltf-test

[andreasplesch]

https://rawgit.com/andreasplesch/x3dom/a3631e31adfe5b3f6552dcf9470068e088551459/test/functional/inline-gltf.html for testing.

[tsturm]

I have start the PhysicalMaterial integration and add support for:

• baseColorTexture
• normalTexture
• emissiveTexture
• metallicRoughnessTexture
• Cook-Torance for direct lighting

Next steps are DDS-Cubemap loading and IBL

Example is updated: https://examples.x3dom.org/x3dom-inline-gltf-2/

[paulmasson]

Examples no longer work in iOS 11: just a blank screen.

[tsturm]

@paulmasson Thanks, I will check it tomorrow!

[tsturm]

For now I disable the old mobile shaders. @paulmasson Can you try again?

[paulmasson]

@tsturm works now

[andreasplesch]

Great progress, almost complete implementation of the default material with a new pbrmaterial node as proposed. Do you want to add Alphamode/cutoff as well ? It may be time to announce gltf2 inline support and the pbrmaterial on x3d-public ? It looks like object space normals as just proposed by Max are also supported/planned ?

[tsturm]

AlphaMode/cutoff should be easy to integrate, but I'm not shure if it's used correctly in all example models. The fins of the BaramundiFish are transparent, but there is no AlphaMode defined in the glTF file and the default AlphaMode is fully opaque.

I think it's not ready for the public, yet. Maybe in one or two weeks, if i'm shure that there is no regression with the rest of the x3dom pipeline.

Yes, object space normal maps are already prepared.

[cx20]

I confirmed that support for PBR was added and I updated the status of gltf-test. https://github.com/cx20/gltf-test I think that it is nearly perfect except animation and IBL. However, it seems that Simple Material and Simple Meshes are not displayed properly. Please confirm.

[tsturm]

Not much to tell today, only some smaller updates:

• Redesign Example Page(more mobile friendly)
• Handling for unlit geometries (no normals)
• PhysicalMaterial fixes

@cx20 Simple Material & Simple Meshes should work now.

https://examples.x3dom.org/x3dom-inline-gltf-2/

[paulmasson]

@tsturm nice new look. Works fine on my iPad.

[cx20]

@tsturm I confirmed that Simple Material and Simple Meshes will be displayed.

[andreasplesch]

https://github.com/bghgary/glTF-Asset-Generator/tree/master/Output Some more gltfs for testing

https://rawgit.com/andreasplesch/x3dom/glTF2TSAP/test/functional/inline-gltf.html

https://raw.githubusercontent.com/bghgary/glTF-Asset-Generator/master/Output/Material_MetallicRoughness/Material_MetallicRoughness_02.gltf

and

https://raw.githubusercontent.com/bghgary/glTF-Asset-Generator/master/Output/Material_MetallicRoughness/Material_MetallicRoughness_07.gltf

ERROR: GL-Error (on searching uniforms): 1282

ERROR: FragmentShader ERROR: 0:46: 'texColor' : redefinition 

INFO: FRAGMENT:
#ifdef GL_FRAGMENT_PRECISION_HIGH
 precision highp float;
#else
 precision mediump float;
#endif

uniform mat4 modelMatrix;
uniform mat4 modelViewMatrix;
uniform mat4 viewMatrixInverse;
uniform vec3  diffuseColor;
uniform vec3  specularColor;
uniform vec3  emissiveColor;
uniform float shininess;
uniform float transparency;
uniform float ambientIntensity;
uniform float metallicFactor;
varying vec2 fragTexcoord;
uniform sampler2D diffuseMap;
const vec4 gammaEncode4Vector = vec4(0.4545454545454545, 0.4545454545454545, 0.4545454545454545, 1.0);
const vec4 gammaDecode4Vector = vec4(2.2, 2.2, 2.2, 1.0);
vec4 gammaEncode(vec4 color){
    return pow(color, gammaEncode4Vector);
}
vec4 gammaDecode(vec4 color){
    return pow(color, gammaDecode4Vector);
}
const vec3 gammaEncode3Vector = vec3(0.4545454545454545, 0.4545454545454545, 0.4545454545454545);
const vec3 gammaDecode3Vector = vec3(2.2, 2.2, 2.2);
vec3 gammaEncode(vec3 color){
    return pow(color, gammaEncode3Vector);
}
vec3 gammaDecode(vec3 color){
    return pow(color, gammaDecode3Vector);
}
void main(void) {
vec4 color;
vec4 texColor;
color.rgb = gammaDecode (diffuseColor);
color.a = 1.0 - transparency;
vec3 _emissiveColor     = emissiveColor;
float _shininess        = shininess;
vec3 _specularColor     = specularColor;
float _ambientIntensity = ambientIntensity;
float _transparency     = transparency;
vec2 texCoord = vec2(fragTexcoord.x, 1.0-fragTexcoord.y);
vec4 texColor = gammaDecode (texture2D(diffuseMap, texCoord));
color.a = texColor.a;
color.rgb += _emissiveColor.rgb;
color.rgb *= texColor.rgb;
if (color.a <= 0.10) discard;
color = clamp(color, 0.0, 1.0);
color = gammaEncode (color);
gl_FragColor = color;
}

[andreasplesch]

https://raw.githubusercontent.com/bghgary/glTF-Asset-Generator/master/Output/Material_MetallicRoughness/Material_MetallicRoughness_04.gltf

no texture, just white

[andreasplesch]

https://rawgit.com/andreasplesch/x3dom/c8c4c9cbdc377298b33352fe93b15c52738553e3/test/functional/inline-gltf.html works with Material_MetallicRoughness_02.gltf : https://github.com/andreasplesch/x3dom/commit/c8c4c9cbdc377298b33352fe93b15c52738553e3

[cx20]

I tried to display a new sample model FlightHelmet.gltf with the library of the latest version of x3dom. However, in x3dom, the appearance of goggles seems to be different from other libraries.

Three.js	Babylon.js	x3dom

[andreasplesch]

Using

https://rawgit.com/andreasplesch/x3dom/c8c4c9cbdc377298b33352fe93b15c52738553e3/test/functional/inline-gltf.html

with the grey background:

All of the inside of the helmet should appear not just the headphones.

The glTF uses alphaMode: blend

https://github.com/KhronosGroup/glTF-Sample-Models/blob/master/2.0/FlightHelmet/glTF/FlightHelmet.gltf#L1117

[cx20]

Sorry, the explanation of the appearance of the goggle seems to be misleading. I wanted to explain that what I see inside the the goggles is different from other libraries.

Three.js	x3dom

In Three.js I can see wood pillar inside goggles, but not in x3dom (I can see headphones).

[tsturm]

Sorry, for the late response.

I switched to a new Branch where I start the integration of WebVR deep into x3dom.

The current version also have some fixes & improvements for glTF:

• some shader fixes
• Normal computation (if missing in the gltf)
• minimal AlphaMode support
• Z-Sorting fix

@andreasplesch & @cx20 Your examples should work now.

https://examples.x3dom.org/x3dom-inline-gltf-2/

[andreasplesch]

Super !

going through the examples here https://github.com/bghgary/glTF-Asset-Generator/tree/master/Output/Material_MetallicRoughness

ok: Material_MetallicRoughness_00.gltf: ok Material_MetallicRoughness_01.gltf: ok Material_MetallicRoughness_02.gltf: ok Material_MetallicRoughness_03.gltf: ok Material_MetallicRoughness_05.gltf: ok Material_MetallicRoughness_08.gltf: dark, but ok

problems: Material_MetallicRoughness_04.gltf: loaded but invisible Material_MetallicRoughness_06.gltf: white, black in 3js without environment Material_MetallicRoughness_07.gltf: rounded corners but should be straight x3dom: reference: Material_MetallicRoughness_09.gltf: invisible, should be white Material_MetallicRoughness_10.gltf: invisible Material_MetallicRoughness_11.gltf: rounded corners, roughness x3dom: bab: 3js:

[cx20]

@tsturm Thanks! I confirmed that FlightHelmet.gltf is displayed correctly.

Three.js	x3dom

BTW, although it is another problem, some models seem to be whitish. Is it because of the setting of the light?

Three.js	x3dom

[cx20]

@tsturm I tested again with the latest version of the library for the following two models. Color of the bear model is now displayed, but Corgi does not seem to be colored. Is the usage wrong?

X3DOM + Bear model result:

X3DOM + VoxelCorgi.gltf result:

[andreasplesch]

Could you try:

https://raw.githack.com/x3dom/x3dom/pbr-shader-work/test/functional/inline-gltf.html

You can click on the upper right link icon and paste in the .gltf url.

The vertex color and base color are multiplied together in gltf.

[cx20]

I have confirmed that the model is displayed properly in your viewer.

It is not displayed correctly in my sample program. Is this due to differences in usage or library version?

[tsturm]

@cx20 You have to use the current experimental build from here: https://x3dom.org/download/exp/

[cx20]

@tsturm Thank you for telling me the location of the library. However, even with that version, the color of Corgi is not colored.

1.7.3-dev - 84ffb8b72710e9c2d93fa8f9b40d03869eb20b07 - Fri Jan 25 20:27:53 2019 -0500

X3DOM + VoxelCorgi.gltf result:

Can you give me some advice?

[andreasplesch]

yes, the pbr shader work branch has an additional fix which needed to be tested.

[cx20]

@andreasplesch Thanks! I confirmed that the problem of color of Corgi was solved in the latest version.

1.7.3-dev - 79a1866a404d6fd494d8dacbc7b245ca36e6a657 - Sat Jan 26 17:54:19 2019 +0100

X3DOM + VoxelCorgi.gltf result:

[cx20]

I tried to display several test models on the following site as additional tests. https://ft-lab.github.io/gltf.html However, the color displayed seems to be different from the assumed one.

X3DOM + Apple model result:

X3DOM + Rocks and trees model result:

The following is the expected result.

[andreasplesch]

BTW, the Corgi model has vertex color attributes for all its meshes which are all white and therefore useless. Maybe three.js has these as a default ?

[cx20]

@andreasplesch The Corgi model in the above was output one year ago using glTF Exporter of Three.js. http://jsdo.it/cx20/CDcF The color specification uses THREE.MeshStandardMaterial and vertex color is not specified. It is unconfirmed whether white is automatically set as vertex color. Perhaps it may be improved with the latest glTF Exporter, so I would like to try it again.