alteous
commented
6 years ago The 2.0 specification is solid. I'd much prefer to see the software ecosystem building out than to see a glTF 3.0 specification.
Open pjcozzi opened 6 years ago
alteous
commented
6 years ago The 2.0 specification is solid. I'd much prefer to see the software ecosystem building out than to see a glTF 3.0 specification.
realvictorprm
commented
6 years ago Is it somehow possible to get a standard for Voxel data?
toji
commented
6 years ago Agreed with @alteous that the glTF 2.0 spec is feeling pretty good at the moment. It addressed some real issues that emerged with the first spec in a very direct and useful way. I'd prefer to see new features emerge as extensions at this point rather than putting too much effort into a 3.0.
I would like to see an emphasis on tool ecosystem improvements. Especially in terms of exporter and other pipeline support. At the moment finding tools that meet a variety of needs is a bit of a coin toss even with some of the Khronos-backed tools. (I recently pushed a small change to the Blender exporter for this exact reason.) One nice thing to have available would be a collection of sample export targets similar to the fantastic set of sample models: A bunch of Collada, OBJ, FBX, etc files that all have different properties which tools developers can use to sanity check their exports. (Did the model with vertex colors come across right? Did the fact that this one has no UVs screw me up?)
Another thing that I'd like to see discussed is a way to verify/sign that a glTF file has specific properties that make it safe to use in a security-sensitive environment when loaded from a third party. (For example: Oculus mentioned using glTF files as favicons in a VR browser.) It's easy in that type of environment to say "We don't support extensions, animations, etc." but it's harder to usefully say "We won't attempt to render objects over 500k triangles" or "We won't load models that are collectively over 10MB" or "We don't support models with more than 100 different materials" which something like a browser would definitely want to do. The idea being that it could be relatively simple for a malicious source to provide a model that contained a few million individual meshes, each made up of a single, transparent, overlapping triangle that each have their own material each with a collection of unique 4K textures which is intentionally aimed at tanking the client's performance. Ideally a model could digitally verify before hand that it does, indeed, contain only 3 meshes with a mere 8k triangles with two materials between them that only use a couple 512 textures, so there's not a GPU in the world that should struggle with that.
That's a Hard Problem™️ , and I can't pretend to even begin to have the answers for it, but if solved it could make glTF a useful format for a massive range of applications that would otherwise balk at accepting external meshes.
Spaxe
commented
6 years ago +1 on "Progressive Geometry Streaming - Fraunhofer POP Buffer, WEB3D_streaming"
jbaicoianu
commented
6 years ago @toji and @alteous took the words out of my mouth. glTF seems like it's in a good place with its featureset, but it's been very difficult to find a glTF asset pipeline that works well. Solid tools for getting glTF data into and out of all popular content creation tools would be a great next step. Some of the older tools like obj2gltf and FBX-glTF desperately need updating to the new specs to be usable (obj2gltf seems like it's making good progress on 2.0 lately, but FBX-glTF hasn't been updated in a year and has dependencies on projects which have been renamed and absorbed into other projects).
Very happy to see blender import support is on the list already - export without import has left me feeling a bit hamstrung sometimes, so this is something I'm definitely looking forward to.
c30ra
commented
6 years ago For software ecosystem I would like to see official UE4 import and blender export. glTF is a good candidate to replace fbx, moreover it's open, so it should have better support in all pipelines/software data exchange.
bwasty
commented
6 years ago For conformance, I'd like to see more sample models, covering the whole spec. I already noticed a few missing things while developing my little viewer:
AlbertoElias
commented
6 years ago Definitely agree that the most urgent issue is improving the ecosystem so that we can all easily make use of what GLTF offers today, which is still a bit too hard to do.
Down the line though, I'd give a big +1 to streaming capabilities, it would be a massive improvement for serving GLTFs on the Web
KageKirin
commented
6 years ago Hi, I recently switched a company internal project from FBX to glTF (2.0). Here are a few tidbits that I would like to see:
Although I agree that the extensions listed above could be easily implemented as custom extensions, I would like to see efforts to agree on a standard for physical data.
Best regards.
pjcozzi
commented
6 years ago This is really insightful feedback, thanks everyone, keep it coming!
To summarize so far:
a compression mechanism for buffers (could be zip or lz4)
@KageKirin check out the draft Draco extension. The bitstream spec has reached 1.0 so we expect the glTF extension to move forward quickly, #874
@jbaicoianu obj2gltf 2.0 support should be ready any day now, would love your help testing the 2.0 branch: https://github.com/AnalyticalGraphicsInc/obj2gltf/pull/67
reduz
commented
6 years ago Please support multiple animations (#1052). Only supporting one makes it pretty limited (and a hassle) for exporting to a game engine. All modern DCCs (Blender, Maya, Max, etc) support animation clips, and all game engines support multiple animations, so this would make the workflow much better.
CentaurMare
commented
6 years ago I've posted both a glTF 2.0 exporter and now a glTF 2.0 importer for Sketchup on extensions.warehouse.com, but both are currently pending approval - not sure how long this process will take.
I'd like to see the specification document improved. I think the minimum and maximum values in the accessors were a difficulty for me, especially as they are conversions from float to string and I was plagued by rounding errors making the validations fail. There seems to be nothing in the document to explain why they are required, other than "there are cases when minimum and maximum must be defined". In the end my minimum and maximum values for normalized values looked like "min":[-1.0001,-1.0001,-1.0001],"max":[1.0001,1.0001,1.0001]
Decomposable matrices were another issue, and a matrix which I thought was decomposable (a matrix built from translate, scale & rotate) would still fail validation - again I don't know if it is because of rounding issues converting from float to string - but it would be nice if the spec included the code to decompose the matrix and what the expected precision and validation requirement would be. In the end I had to give up and just pre-multiply all the geometry by the transformations first, so exporting with identity matrix only for my version 1.0.0 exporter.
Other parts of the specification could have a bit more of a tidy up, e.g. "When nor sparse, neither bufferView is defined..."
donmccurdy
commented
6 years ago Please support multiple animations (#1052).
I've posted more details on #1052, but this is already supported — our examples do not make it obvious. Definitely +1 from me on changing the example models to not use different animations for each node, and adding some examples that have multiple distinct animations.
fire
commented
6 years ago What is the status of hdr formats? Openexr would be standard, but I'm open to ideas.
Openexr supports both 16 bit float and 32 bit float which corresponds to OES_texture_float; OES_texture_half_float.
erwanmaigret
commented
6 years ago Actually in general the missing bit to me is more about being able to add extensions/plugins. I strongly agree that 2.0 is already very strong and doing a lot, and going too deep may start to make it too specific (let's not make it grow into an FBX-style format).
So my main suggestion for extension would be to introduce some type of standard plugin solution that could allow custom extensions to be added and embedded inside gltf data.
As a simple example the rigging part is pretty standard today to map what you can do in Unity, but if you are in need deeper notion of constraints or deformers then you're stuck with only skinning/shapes.
The problem being that plugins mean evaluation mechanism during playback, most often seen as a complex problem. I actually think it's pretty simple if we could have a notion of scripted based behavior with embedded js or something like this (call it scripted operators, unity behaviors, or whatever). And avoid the big language debate by making this js-only since it's targeted mostly for web.
And btw, neither FBX now dot.xsi / KL / ... really tackled this right since they are all biased as companies, we have the luxury to make it right, and this could grow a huge community of developer IMO.
Noxime
commented
6 years ago @erwanmaigret
I don't think adding a scripting language to a model format is a good idea, especially JavaScript. What I would rather want to see (I haven't really read spec, so this might already be supported) is arbitrary data. Meaning you could embed anything you want to the file, possibly in base64 encoded binary.
toji
commented
6 years ago @Noxime: That already exists. Buffers and BufferViews in glTF are references to completely generic binary blobs which can contain anything you want in any format you want. If there's data that's unique to your use case you can easily create an object in a relevant extras property somewhere that points to a BufferView and provides whatever metadata is necessary to interpret it.
c30ra
commented
6 years ago @noxime
Both javascript and arbitrary data are bad idea imo because this way you're are creating a vector of possible virus/exploit specially on pour coded library. With script you basically allow execution of arbitrary code, and arbitrary data may cause exploit by using particular string(something like happened to WhatsApp some time ago).
For conformance: Also, I don't like much the idea of "extensions": Having an extension mean that an implementer may or not support it. This mean that a software may implement it and another one no. In case one want to pass data from this software to the other it virtually lose information. Instead a non extension spec, must be implemented to be compliant with the implemented version. There is also the problem of bloat the format with less used or exotic stuff..
donmccurdy
commented
6 years ago More information on glTF's extension system, and why we have it, here. Extensions might specify extra data, intended to be used in a particular way by the runtime, but will not include scripts implementing that behavior.
Implementation of an extension's behavior may be handled by extending the loader for a specific engine, rather than as part of the glTF format itself. For example, we (three.js) are hoping to allow users to write plugins for THREE.GLTF2Loader without having to modify three.js directly (https://github.com/mrdoob/three.js/issues/11682), so that you can experiment on extensions as needed.
donmccurdy
commented
6 years ago Also, I don't like much the idea of "extensions": Having an extension mean that an implementer may or not support it.
The intention here is, features that can and should be supported by all implementers will be part of the core specification, not extensions. As such, content authors who want their models to run everywhere should stick to the core spec.
However, we don't want to limit glTF so much that it only supports features possible on low-end mobile devices. For more expensive materials (like PBR specular-glossiness, or SSS) or WebGL 2.0 features, we hope to use extensions to enable experimentation, and then these features can be brought into the core spec if/when the time is right.
erwanmaigret
commented
6 years ago I totally agree bringing some language in the game is far from ideal. But then I don't see any other options unless we start to get into predefined constraint+deformer notions.
Both could work in the end. I am afraid that if we don't do either, then the format will be limited to embedding only skinning/shapes/animation forever and side solutions will start to be created.
Let's just consider a very simple aiming constraint, which is very commonly used today in about any rig with a character. How could we make it so it's part of the data part?
Or maybe this just does not fit into the low-level gltf format but then we should consider specing out an official solution as a standard gltf behavior manager?
stevenvergenz
commented
6 years ago I disagree that glTF needs an embedded behavior system. At the end of the day, glTF is specifically a data transmission format. It's the jpeg of 3d, not the javascript. I think it would be useful to be able to serialize and transmit "entities" instead of "models", but trying to shoehorn that into glTF is a mistake. That deserves its own spec IMO.
toji
commented
6 years ago @erwanmaigret: Things like aiming constraints sound highly use-case (and probably engine) specific. That's what the extras attributes are for. Certainly there's nothing preventing you from putting a full scripting language into an extras attribute if you wish, and your content pipeline can be tuned to use that, but everyone else loading the file will only use the parts of it that are standardized. That sounds problematic, but in reality there's very little chance that something as specific as an aiming constraint will map cleanly between environments anyway.
erwanmaigret
commented
6 years ago @toji Make sense I'll write this as a rigging/behavior engine riding on top of gltf then in a separate repo.
pjcozzi
commented
6 years ago (Minor summary update; italics indicates updates from https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-320047171).
steveghee
commented
6 years ago Personally, I'd like to see focus on complexity management e.g. supporting levels-of-detail (and streaming), being able to support structured breakdown (json file referencing other json file(s) - #37 ) and finishing up the alternate material support e.g. a simplified blinn/phong model.
alteous
commented
6 years ago For the sake of simplicity and ease of adoption, I hope that the scene hierarchy remains a strict tree. I say this because the nodes and heirarchy section of the specification mentions that this restriction may be lifted in the future.
steveghee
commented
6 years ago I would assume that if this is going to be truly platform neutral, there is going to a cleanup of some of the GL-specific enums e.g. componentType : 5126 (GL_FLOAT).
xelatihy
commented
6 years ago In case you care, https://github.com/xelatihy/yocto-gl has a C++ loader/writer and example code for both a small viewer and OBJ2GLTF. All C++ and tested on all models from the glTF 2.0 spec.
xelatihy
commented
6 years ago As for the new spec, I would suggest
Let me comment a bit, and please consider the fact that I am mostly interested in C++ and fast execution speed. glTF might be a great fit for JS and WebGL, topics on which I am very ignorant about. 1, 3 and 4 can be done with extensions, but 2 cannot. But extensions are not that useful since tooling won't really follow them.
glTF has a ton of indirection that I would love to see removed. For example a. mesh -> shape -> accessor -> bufferview -> buffer -> data b. material -> metallic-roughness -> texture_info -> texture -> image -> data or buffer All of these slow down native code and are sources of errors for data files on the web.
Also, for our projects, it is very problematic that buffers are not typed. When we work on asset we can easily work on images using PNG. PNGs can then be shared with different glTF without requiring modification to the JSON data.
For meshes, we cannot do that since the spec forces us to change all glTFs that reference a buffer. But if I just want to add data to a mesh, like tesselate, I should not have to change all scenes that they are referenced into. Also, we can easily work on image files in parallel, but cannot do so on meshes again since all changes needed to be manually reflected in glTF. This makes a lot of asset processing way more complex than it should be and makes meshes not behave the same as images.
xelatihy
commented
6 years ago BTW, as a testament that glTF indirection are hard to handle in native code, all native libraries I look at create alternative representations for geometry and do not allow for accessing glTF natively. Again on the Web maybe all these indirection are good. But in native code they are hard to handle. Also the mesh -> shape -> bufferview -> buffer -> data makes it really hard to use glTF has intermediate for editable files (I know that it was not meant to be, but less transcoding is always better).
toji
commented
6 years ago @xelatihy: I don't think that the indirection is there for the benefit of the web, per say, but is instead a factor of trying to model the needs of a graphics API fairly directly. In many cases you can see a 1:1 parallel between a level of the hierarchy and a call that needs to make to set up an OpenGL/WebGL draw call. Other graphics APIs may not map quite as directly but will still need the same data.
The indirection also allows for efficiency in how data is packed in many cases. For example, glTF could easily represent a scene where all of the objects share a single vertex/index buffer but are split up across several draw calls for the sake of changing materials. I can't think of any other formats that allow for that kind of optimization off the top of my head, even though it's a pretty common thing to do when rendering.
This also accounts for some of why many loaders will re-structure the hierarchy for rendering: The optimal bind/draw patterns vary based on a lot of different factors, and frequently won't match a hierarchy who's primary concern is logical reference order. glTF files can imply instancing through the node tree, for example, but if your renderer actually wanted to use hardware instancing in those cases you'd need to cache the data slightly differently. Beyond that, most existing code bases will have an internal structure for their graphics data that nothing but a custom built format would match. Obviously such scenarios are going to require transcoding.
Overall I've been pretty happy with how minimal the glTF JSON hierarchy is. It's exactly enough to efficiently express a structure which minimizes the number of files and buffers referenced.
I do see what you're getting at with the external mesh file request, though I would phrase it slightly differently since it took me a moment to understand the request. I think separating the format into "scene" and "mesh" files would be a bad idea, but I can definitely see a use case for allowing a glTF scene to include an externally defined scene file as part of it's node tree. At that point an "external mesh" is simply a glTF scene with a single node that points to the mesh in question, and you could have a separate glTF scene that builds out the actual scene hierarchy with the external files linked at the leaf nodes. May be worth introducing a single level of indirection to make it efficient to instance an external scene multiple times within the hierarchy.
xelatihy
commented
6 years ago @toji I see that the design was mostly WebGL based. But it seems that glTF might support more use cases and the coming of Vulkan/Metal/NextVersionOfWebGL, might make some of the initial concerns less stringent.
On native code, the format is a bit hard to handle, at least by looking at the C/C++ libraries out there. My experience so far has been that if you have to process the geometry on the CPU the indirections and the loosely typed system make it not so easy to use for data processing. So most libraries lift out the data in another representation.
As for having separate mesh formats, it would make processing them easier, just like processing PNG does not require to load the glTF scene. Note that this could just be embedding a bit of JSON before the buffer data, just like binary glTF. This is similar to images where I can just add a new file format and all of the JSON structure remains the same. There are also application that benefit from having meshes separated, e.g. Pixar's USD does it for example.
Having said that, glTF works already quite well. Just trying to suggest that maybe a few small modification might make it more efficient in other application domains.
CentaurMare
commented
6 years ago For glTF to achieve widespread adoption across multiple industries, it needs to be easily implemented.
As a simplification, I'm wondering if primitive.mode should be limited to just triangles (mode 4) and lines (mode 1) and the other values deprecated. I don't see the usefulness of such types as triangle strips and triangle fans - and while they may have been useful in OpenGL in the early days, I don't believe a model exporter would ever create anything other than basic triangles and lines. Sure it is easily implemented in a client engine, but what about importers? Should they need to deal with the other possible modes?
Likewise, non-indexed geometry should be deprecated. Is there really an existing modern exporter that will generate non-indexed geometry for drawArrays? Non-indexed geometry is just another case that an importer needs to consider when attempting to import the widest range of glTF models possible. I do see the case that export writers can benefit from the initial simplification, but allowing it could just encourage poor export implementations creating bloated models.
Edit: The normalized UNSIGNED_BYTE and normalized UNSIGNED_SHORT for UV coordinates, etc., could also be deprecated if glTF is to emphasize TF (Transfer Format) rather than gl.
Moguri
commented
6 years ago @KageKirin A physics extension has been started here. Perhaps it can be expanded to include some of the features you would like to see?
pjcozzi
commented
6 years ago Thanks for the great input as always!
For the sake of simplicity and ease of adoption, I hope that the scene hierarchy remains a strict tree. I say this because the nodes and heirarchy section of the specification mentions that this restriction may be lifted in the future.
@alteous in the pre-1.0 days of glTF, this was a DAG, which generally required converting to a tree at runtime. Since this level of complexity in the runtime was not in the spirit of glTF, we decided to require a tree. The note is the spec is in case anyone comes up with some important use cases where DAG -> tree should be done at runtime instead of offline. Perhaps unlikely.
I would assume that if this is going to be truly platform neutral, there is going to a cleanup of some of the GL-specific enums e.g. componentType : 5126 (GL_FLOAT).
@steveghee this was discussed at length; glTF 2.0 stayed with the GL enums to minimize breakage from 1.0 and since no set of enums would map perfectly to all APIs, so at least this maps well to GL. The D3D implementers were OK with this.
pjcozzi
commented
6 years ago (Minor summary update; italics indicates updates from https://github.com/KhronosGroup/glTF/issues/1051#issuecomment-320389530).
pjcozzi
commented
6 years ago We would also love input on how to handle validation/conformance for exporters? What is pragmatic? What will help exporters always output validate glTF?
Input please...even if it is just +1/-1 for some items. Thanks!
Kupoman
commented
6 years ago For exporter development, I would love to be able to run a bunch of samples through my exporter and then through a validator to set up some kind of continuous integration. The validator part is mostly there, though it looks like it does not validate glTF 1.0. The test files are more of a problem since Blender does not have an importer to load the glTF sample files. I have not yet looked into converting the glTF samples to something Blender could import, but having them available as something like OBJ could be nice. It would also be nice to have a robust set of test samples to target specific features to use them kind of like unit tests.
In short, as an exporter developer I would love a robust test suite of sample files available in multiple formats (or tools to convert to other common formats) and a thorough validator.
CentaurMare
commented
6 years ago We already have everything we need for creating an exporter.
What really assisted me while creating an exporter in Ruby was the glTF Validator (http://github.khronos.org/glTF-Validator/) and Don's Viewer (https://gltf-viewer.donmccurdy.com) once I had something that passed validation.
I initially struggled with the basics, and I returned time and again to the 'simple meshes' and 'triangle' example models.
It may be of use to provide the same models but with a texture, and also simple models with multiple primitives (two or three materials) as the larger models can be a bit overwhelming. Multiple Primitives is even something Microsoft failed to grasp with their implementation of binary glTF for Paint3D, so the spec should give more examples, and not just leave implementation examples to sample models.
As it is an exporter that is being written, whatever source models in the application (in my case, SketchUp) can be used, but what would be useful would be to know that the final glTF model is the correct dimension and orientation. So maybe the glTF validator could show the final bounding box for the entire model, and that could be compared with whatever model is in the application as a quick test.
steveghee
commented
6 years ago i would like to see continued emphasis on compression, so unsigned byte, short and half float representations for vertices, normals etc. are very useful. Tristrips and fans are useful for certain surface types - i know we could use them in our export from CAD.
McNopper
commented
6 years ago @CentaurMare Blender glTF 2.0 exporter has many, basic examples: https://github.com/KhronosGroup/glTF-Blender-Exporter/tree/master/scenes You just need to export them.
Marlamin
commented
6 years ago I agree with @CentaurMare, the validator and Don McCurdy's viewer are basically required tools in testing exporter output. Dragging and dropping a folder/zip onto the viewer feels very exciting every time you've fixed a bug/added a feature.
Two things that I would personally like to see get higher priority:
Thanks everyone for their work so far, glTF has been a dream to work with compared to OBJ and the short time I've spent looking at FBX.
kimkulling
commented
6 years ago In Asset-Impoorter-Lib we are using a set of models including default feature, whcih are supported by the exporter like light-definitions for example. And we are also providing a set of edge-cases to ensure, that older bug does not occur in the future again.
We are using the following algorithm to validate the export:
When decetding a mismatch it make sense to generate a diff between the older version ( for instance by storing the older executable inside of the archive ) and the new version of the exported model. So debugging the issue will be much easier.
zellski
commented
6 years ago I really like glTF. You are wonderful people. This is a splendid project.
Off the top of my hat:
CentaurMare
commented
6 years ago @steveghee While I concede that unsigned byte, short and half float representations for vertices, normals etc. are useful for compressing the final glTF file, from an importers point of view, it is extra coding and testing that needs to be implemented. Also, glTF is relying on compression provided by gzip in the web transfer, and I believe this should be sufficient for compression, but again, I think this issue would need to be further studied. My preference is that non-float representations should be deprecated.
Tri-Strips and Fans should, again in my opinion, only be exported as an indexed mesh. It would place too much burden on the importer to support yet another mode. Yes it is easy for the exporter in certain cases, and it is certainly easy for the OpenGL implementation to render, but it is not easy to implement for the importer.
We want glTF to be widely used in industry, not relegated to niche environments because it is too complex to implement.
lexaknyazev
commented
6 years ago @zellski
The validator project confuses me. There seems to be a 'snapshot' branch wherein a dart executable validates 2.0 (and I use this often), while the 'master' branch contains source code that validates 1.1? Does this mean source is being developed elsewhere, in some private repository, and binary drops committed to the public one? This is confusing and possibly disturbing. Ideally I would love tags for a 1.0 validator source, 1.0.1 validator source, 2.0 validator source.
There're some historical reasons for this situation. Initially, Validator project was developed for glTF 1.0 but due to certain spec issues it was retargeted to 1.1 (renamed from 1.0.1) version of the spec. glTF 1.1 was never released since we moved to 2.0 with PBR instead.
Full 2.0 validation source code will be published in 1-2 days.
lexaknyazev
commented
6 years ago Tri-Strips and Fans should, again in my opinion, only be exported as an indexed mesh. It would place too much burden on the importer to support yet another mode. Yes it is easy for the exporter in certain cases, and it is certainly easy for the OpenGL implementation to render, but it is not easy to implement for the importer.
@bghgary @sbtron Thoughts?
zellski
commented
6 years ago @lexaknyazev Thanks, that makes sense. Do you think there's a commit hash that one could tag as v1.0? I know 1.0 is thoroughly deprecated, and I understand nobody wants to put more work into it, but there are still occasions where it's useful to validate it, and if there's a version that worked, even if it wasn't feature complete...
lexaknyazev
commented
6 years ago @zellski
Do you think there's a commit hash that one could tag as v1.0?
Unfortunately no. One of validator's goals was to tighten up 1.0 spec, so Validator was never meant to verify initial 1.0 version (we were accumulating little issues into 1.0.1 which became 1.1 due to noticeable changes).
Hi all - please chime in with any and all feedback to help drive the direction of glTF beyond 2.0. Even simple +1/-1's for topics are appreciated.
How much should we focus on building out the software ecosystem vs. moving forward the spec?
Should new spec features come in as extensions first, direct spec updates, or some combination? What mix?
Ecosystem Tools
What glTF software do we have the most immediate need for?
Conformance
Infrastructure
Learning Material
Potential spec updates / extensions
Really looking forward to continuing to move the field forward! Can't wait for your input!