states

[RemiArnaud]

A technique can have states. But what do they signify exactly?

1) the difference between the WebGL default state and the state needed by this technique? or 2) the set of states and values that are important for this technique

In order to do lazy state management, and minimize the number of WebGL calls, I would be better if if was the second case.

in the first case, one has to know what the default state values are, and when applying a technique, it has to reset the state to the default value if they are not listed in the technique. But there are several states that a technique does not care about at all. So this is wasteful.

In the second case, only the states listed in the Technique need to be checked, and set if different. In the second case even if the state is the same as the default state value, it has to be listed explicitly in the Technique.

[pjcozzi]

CC #85

[pjcozzi]

We should handle this like any other part of the glTF spec. states contains all the fixed-function states, each of which has a sensible default if it is not included. Since the sensible default would be to match the WebGL default, this falls under (1).

I believe you are tying this too closely to your implementation. In my engine, and I suspect in the Three.js renderer, (1) is easier to integrate than (2). (1) is also more cohesive and easier to understand.

[fabrobinet]

I was also about to ensure we implement 1). Converter work to get default values per profile..

[fabrobinet]

2) the set of states and values that are important for this technique

Is a bit vague as a definition, != from default value is indeed clearer - I agree with @pjcozzi.

[RemiArnaud]

If it is (1) -> differences from default values, then we need to add 'dontcare' for the states that do not need to be reset, that have no influence on this particular material.

here's why: With current gltf State A is currently not at it default value. New states do not include state A. This means that state A has to be reset to its default value. Repeat for ALL glTF states.

with (1) + don't care State A is currently not at it default value. New states says 'dont care' for state A. Run-time has nothing to do and keep State A as-is. Repeat for ALL glTF states

with (2) State A is different than new state. Run-time has to set State to new value. Repeat for ALL new states (which is less than all states)

I prefer option (2) since the state management loop is smaller. We could add 'don't care' to our current definition. But our current definition is definitely not optimal as far as state changes

[pjcozzi]

We can't assume anything about how an engine implements state management/sorting. I suspect most serious glTF implementations are going to be built on top of an engine that already has abstractions for render state and just needs to map the glTF states to its data structure. The role of glTF is to cleanly and efficiently transmit the data, not to tune for one implementation strategy.

At best, I could see this as a proposed extension, but I do not see it as part of the core glTF 1.0 spec.

[RemiArnaud]

Please take the time to read this again. It has nothing to do with how an engine will implement the state management. It has to do with lack of information in glTF for any implementation to do a good job at it.

[fabrobinet]

As an aside I tried, but it is a bit hard to read :).

Just the first sentence sounds wrong to me:

If it is (1) -> differences from default values, then we need to add 'dontcare' for the states that do not need to be reset, that have no influence on this particular material.

Every state have an influence (including all default ones), that's why we would like to just surface the ones who are not equal to the default GL state.

Taking (2) as you mention:

State A is different than new state. Run-time has to set State to new value. Repeat for ALL new states (which is less than all states)

(*) What if a state was set previously in technique A and not set in technique B. How do you handle this ? I guess you don't want to specify all gl states, so are you suggesting we you should the set of all used states (non default) in a given asset and set them all for all techniques ?

Also, Just sticking with non-default values you can maintain within your renderer a list of all used states and reset the ones not used for a given technique, and same applies to your (2) unless you want to implement what I suggest above see (*), but it wouldn't be great because it would carry potetentially more states than you actually need to care.

The bottom of line to me is as @pjcozzi . We provide the states that are the ones you need to set because they are not GL default. This way, we just build incrementally on top on existing GL SPEC (just declaratively) and we do not invent new rules when we don't need too...

[RemiArnaud]

"I guess you don't want to specify all gl states, so are you suggesting we you should the set of all used states (non default) in a given asset and set them all for all techniques ?" Could you rephrase, I don't understand this sentence.

to summarize, what I am looking for is how to handle to following:

Given a current gl state (shadowed), the default states (as in the spec), and the list of states in the technique. What is a run-time supposed to do to set the proper gl state?

and more specifically, for a given gl state ( say LineWidth for example) 1) gl state is same as default state, and not in the list of states of the technique 2) gl state is different as glTF default state, and not in the list of states of the technique 3) gl state is same as default state, and is in the list of states of the technique

[fabrobinet]

sry for typo: "I guess you don't want to specify all gl states, so are you suggesting that we should set all used states (non default) in a given asset and set them for all techniques ?"

[RemiArnaud]

Can someone answer the questions above please?

[meshula]

Here's what I would expect

1) gl state is same as default state, and not in the list of states of the technique -- run time does nothing

2) gl state is different as glTF default state, and not in the list of states of the technique -- run time must set glTF default state (because otherwise materials are cumulative and order matters - in other words to be in state (2) the previous material set a non default state that did not get cleaned up)

3) gl state is same as default state, and is in the list of states of the technique -- run time must set state specified by technique if that state is not the default state otherwise run time does nothing

Basically technique must be a delta from default state is my reading. That is how I always implement material systems, so that is how I want to read it :)

[RemiArnaud]

thanks, that is what I would also expect.

Now, please consider (2)

"2) gl state is different as glTF default state, and not in the list of states of the technique -- run time must set glTF default state (because otherwise materials are cumulative and order matters - in other words to be in state (2) the previous material set a non default state that did not get cleaned up)"

Some states have absolutely no influence over the material/draw. For example, LineWidth has absolutely no influence if there are no lines to draw; blend parameters have no impact if blend is not turned on... and so forth.

the only way to not reset states to default values in those cases, is for glTF to add additional information to the state as proposed in the first post in this thread.

@pjcozzi , @fabrobinet - does this make sense now?

[fabrobinet]

I agree with that from @meshula and I believe that's what I have said earlier too:

Basically technique must be a delta from default state is my reading. That is how I always implement material systems, so that is how I want to read it :)

I still find that the concept of what is "important" to a technique is very vague. I get your examples above, with lines and blend parameters, but it's also potentially, error prone (and a bit clumsy) to have to set a default value explicitely in the tecnique because such state would be "considered" important, for instance, it means BLEND (enable/disable) would have to be always set. If I got what you suggest.

Also, it doesn't answer my question earlier about how to handle if an "important state" for a technique A is set and then another technique B with different "important state" are set. You will still need some cleanup between the 2 techniques to set default values that needs to be set and not tracked by technique B.

So all that together, I am still for a simple thing: Only specify states that aren't the default ones

[RemiArnaud]

‎Please explain. I don't understand with the current information in glTF how I would know that the 'lineWidth' state has no impact, and therefore I do not have to change its value, regardless if that value was changed from its default value. 

[fabrobinet]

it's the up of the client engine to adopt any strategy to maintain the default state in this case.

One possible way is using a set (as a collection) of updated (and thus) non-default states can be maintained... what's left in this list once you have set your states are leftover from the previous passes and you can set them to default and clear the list.

Again - with your proposal you got same kind of management to do when switching from one technique to another without the benefit of a clear definition such as: "only non default values in states".

[RemiArnaud]

‎"One possible way is using a set (as a collection) of updated (and thus) non-default states can be maintained... what's left in this left once you have set your states are leftover from the previous passes and you can set them to default and clear the list.‎" 

This does not answer the question still. The question in not how to keep track of states and reset the one that have been changed to default values. The question is how not to care about some of the states if they have no impact at all.

[fabrobinet]

If some states have no impact, why would we need to care, and why would we set them. ? Maybe simpler would be to meet and solve that in French :).

[RemiArnaud]

‎Exactly. Now we do not have that ‎ information currently.Currently we have to reset the state to its default value, we do not have the information that we do not care about this state.  -- Rémi

[fabrobinet]

You can know about this default value, by querying your GL context at init time or looking up at OpenGL spec. I answered that earlier in this issue. If you wanted to never have to care about what is the default value of a given state, you would have either to provide all GL states each time, but that's not acceptable, or gather every possible state that is non default for every technique, and for all technique set all states that could have been possibly changed. Which makes potentially a lot of false positive.

[RemiArnaud]

Still not addressing the problem. The question is not how to know what the default state is. The question is: how do I know that I should not reset a state it it's default value when it's value does not matter. -- Rémi

[fabrobinet]

So in some way.. you would be enumerating all states that you believe are "important" .... Could be in a separate list, or with all states with default values. But you want to list everything that needs to be taken into account for pass. Wether it would be at its default state.

Quite risky... I find that desire very hazardous.... Typically when you write shaders, you think of what needs to be enabled. Or you figure out that needs to be enabled :)... It's not the same effort to worry about the whole GL state machine and be sure you "declared" something that may impact rendering even if it is silently still in its default value.

Really, I believe what you suggest would be too much error prone and unintuitive as it requires too much care to make sure that all GL states are potentially impacting a shader are either enumerated in some way in each pass.

[RemiArnaud]

‎Yes, absolutely glTF has to provide all the necessary states for correct rendering.What we are missing is a way to know about the states that have no impact on the rendering. This can be done several ways. A list of 'don't care' states is a possibility. Having the states list all the necessary states - even if same as default state is another solution (this list would not include the states the rendering does not care about). Or any other solution, it does not matter how this information is available, as long as it is available so the rendering can make optimal choices. I don't understand what you mean by error prone. Currently someone has to list all the states that need to be set if different than default. What is the problem to also allow listing states that are not necessary to check?  And I we have not yet addressed the issue of state inheritance. I understand from the comment in the 'texture' thread that a state can be set in the technique, and that state is then now the default value for the material? ‎ -- Rémi

[pjcozzi]

What is the problem to also allow listing states that are not necessary to check?

Because many engines will not need this. We can propose this as an extension but this is not needed for the core spec. I understand that your engine can use it for optimization but most will not care because they have more general state sorting already implemented.

I understand from the comment in the 'texture' thread that a state can be set in the technique, and that state is then now the default value for the material? ‎

State is just part of the technique; there are not overrides like how technique parameters override material parameters because material doesn't have a render state property.

[RemiArnaud]

"Because many engines will not need this. We can propose this as an extension but this is not needed for the core spec."

In that case we should remove things like names from the core spec as most existing engines do not‎ use that information. In fact names are not used at all by engines, only if the value need to be shared to the end user. Note that I am not advocating to remove names, I am just taking this example to show how arbitrary is the new rule of design you just used to pushed away this state issue. 

"‎ I understand that your engine can use it for optimization but most will not care because they have more general state sorting already implemented." 

Are you implying that most engines do not need that information because they can somehow create optimal state sorting somehow without that information? There are no possible algorithm that can create optimal state sorting without knowing what states are not important for a given rendering. So I have a hard time to make sense of what you mean there? if you can explain please. Anyways the matter is that glTF does not provide enough information for any engine to do optimal state management. There are several way to add this information in glTF. I really have a hard time to understand why we would not want glTF to enable optimal state management. Why limiting glTF to sub-optimal state sorting applications?

I understand from the comment in the 'texture' thread that a state can be set in the technique, and that state is then now the default value for the material? ‎

State is just part of the technique; there are not overrides like how technique parameters override material parameters because material doesn't have a render state property.

[RemiArnaud]

I believe the issue is still not understood

To better illustrate the problem at hand, here's a simple state management question for someone to answer:

Example:

StateA: default is false
MaterialA: states={stateA:true)

What would be the optimal state management ? (assuming state is default value when we begin) Answer:

 Set stateA to true
 Draw

Note that for subsequent frames it is not necessary to apply stateA again, since it is already set to true.

Question: starting with default state, with the following information in gltf, what is the optimal state management for those two materials/draw?

 stateA: default=false
 stateB: default=false
 MaterialA: states={stateA:true}
 MaterialB: states={stateB:true}

[fabrobinet]

Set stateA to true
Draw
[cleanup whatever needs to be (*)]
Set stateB to true
Draw
[cleanup whatever needs to be (*)]

(*) all depending on engine implementation.

Since a few comments I think I understood your point. Correct me if I am wrong, but part of what you call "optimal state sorting" is having a way to possibly let "dirty" a state (here stateA) if MaterialB is not "impacted" by it. That's what you suggest with the "don't care" list, and I maintain that it is not something I find appealing to specify. A state like BLEND, or CULL it is very likely to be always "important", so considering or not their importance sounds a bit moot to me. Now, for other examples you provided, like lineWith you suggested, if the model being drawn is not made of lines then you don't care, you don't need a list for that, and same if you had blending parameters but blending is not enabled you don't care either, and not need for a list for that either.

Starting simple by specifying only what's not default may not be as optimal as you'd wish, so you can find it frustrating, but it is unlikely to bring us legacy. As @pjcozzi mentioned, if later on, we collect more feedback converging to your suggestion we could start with an extension and eventually add a property.

[RemiArnaud]

@fabrobinet , unfortunately your answer is not helping moving forward establishing the issue. Especially that step:

[cleanup whatever needs to be (*)]

It is unfortunately masking the exact matter that we need to discuss. glTF needs to be clear on how state management is working. It is indeed up to the implementer to decide how exactly it will be implemented. Your answer in fact illustrate the need for this discussion, as depending on what the 'whatever needs to be' step does, it may or may not provide the desired result.

So, is it possible for someone, to answer the question without referring to unspecified steps that should do the right thing....

Once we have establish the issue at hand, we can discuss how to solve it. I see your previous answer some interesting elements that would need a lot more work to be complete though. Let's first establish the problem clearly

Thanks

[dlannan]

Having an interest in where glTF is going and what it might be doing, I'd like to ask a simple question that should help find a solution to your "state management".

Is glTF going to manage all the states of GL? If so, why? Normally, as others have mentioned, states are managed by the implementation side, not the interface.

The main reason you should not have state management in glTF is that different implementations have different requirements on how states should be managed. A pure 3D game rendering system may want specific render state optimisations done (which they should do). An application using shaders for mathematical use (ala CUDA / OpenCL) will have little or no use for any state management.

In my opinion, do not manage something that a user may need to do so themselves.

[fabrobinet]

@RemiArnaud I proposed one way here a month ago: https://github.com/KhronosGroup/glTF/issues/249#issuecomment-37986263 Let's meet when I come back from paternity leave next week and discuss implementation strategies in more details. I believe we reached a point where a meeting would be more productive than discussing via github issues. And we'll of course put there our conclusions.

@dlannan if you have already a game 3d rendering system, you may not even use shaders and just use the details property in passes that contains all the meta informations to regenerate the shaders and states. But for the ones who want no big dependency on some 3d engine and just display easily an asset, then providing all the states and shaders lower the barrier to be able to display a 3d asset and I think it has value.

[dlannan]

@fabrobinet - Pardon my ignorance I have been working on putting together a live 3D simulator in WebGL (ThreeJS + Meteor + AmmoJS), and I am looking at glTF as a data transport - hence why Im now very puzzled. Surely the goal of glTF is to provide a data set that can be used by a consumer application (3D engine, or whatever) as needed. glTF as a spec is not defining how that data is to be used or managed? Its defining a data structure? So in your example isn't the viewer application responsible for how to "just display easily an asset" and to handle state? I'm puzzled how a data interface should even be managing state - from my simulator perspective the glTF data set is an asset delivery structure for visualisation. How my renderer is implemented is of no consequence to the glTF format, assuming it contains all the data needed to render the original data set. This is why I am puzzled - state should have been set when the source 3D asset was generated, then how that data is then interpreted and used is again, entirely driven by the output system. If glTF is not like this, then please let me know, since I'm specifically looking for a good transport layer for WebGL - currently CTM is the other option I am working with.

[fabrobinet]

Say you have double-sided faces, you want to disable face culling, say you have transparency, you will enable blending. E.G You can't escape setting gl states at runtime. So my point above was just to say that to ease rendering assets, all these states are declared in glTF. This way adopters can just write a simple data driven engine that handle gl states.

In your case, you have a higher level engine that certainly is able to set the right states based on higher level properties like double_sided or transparency, so you probably don't need/want to use the provided gl states. It is fine. You can still rely on higher level informations. As I mentioned above with the details property.

[fabrobinet]

also @tparisi made a glTF loader that has landed in the dev branch of ThreeJS. And I guess he doesn't care about the states, he just look at the high level properties.

[dlannan]

Im not sure what state decision glTF can make? Put simply - our 3d artist sets the state to make the result look correct. As far as im concerned glTF should simply reflect that. There should be no point where glFT decides state. Otherwise how does the artist ensure correctness? It seems to me you are making too many decisions which is not a data structures role. I think I'll keep working with CTM. Im really not keen on another messy collada type data set for web deployment. Best of luck with it all - imho it seems like you are mixing too much function with data which will complicate systems that need to use it.

[fabrobinet]

When you are using Three.js the exact same steps to set gl states are taken. You could consider equally that Three.js is taking "decisions", but in both cases the way you are wording that is not accurate. Both glTF and Three.js translate high level concepts from material to GL. E.G It is just hidden for you in Three.js, glTF has it explicit because it does not mandate a high level engine to be usable but again this state handling is not mandatory (if you use the details property). I don't want to convince you, if you are fine with CTM, it's good, but your arguments aren't accurate.

[fabrobinet]

also, when you talk about "our 3d artist sets the state to make the result look correct" it shows that we don't talk about the same thing... gl states discussed here and material definition (set by artists) are at very different level of abstractions. (assuming it is not raw gl states you set in your tool...).

[dlannan]

Sorry - Im not sure what your 3D artists do, but ours explicitly set glstates for all sorts of things. If you have ever used Ogres Ofusion, or other Material extensions in 3dsMax you explicitly set glstate flags and values (from ZDepth, ZWrite, LineSize - as mentioned above). The artist must do this for many reasons in our scenes (even in Unity you can set explicit states) - setting layer rendering flags for example. These are directly output to Techniques in Ogres shader file format and in Unity into shaders and materials. So, no, Im sorry but you are incorrect that artists do not do this. And in the game industry it is even more so - at Pandemic Studios and at other places we all used tools to let the artist exactly define how a material and texture should look. Just because a state exists in gl means nothing - this is a data value that must be passed from the artist to the output. Whether its DX, GL, PS2, Xbox states.

Also, ThreeJS IS the engine. Hence what I am explaining is quite accurate. The loader IN ThreeJS loads the glFT format - this is the representation you are defining. This data representation is for 3D assets, not a functional description of how those assets work. The engine defines how that data is executed. If you dont agree with this, then I really dont know what you are trying to build - to me it sounds like you want to make glTF an engine rather than a data format.

[fabrobinet]

OK I see, then in your format you have gl states set by artists, exactly what glTF consumes. Great ! These would be just preserved and serialized in glTF I don't see the problem. How to land this info from the tool to glTF could be done in different way but it is another topic, just focusing on the format, if you provide states it's great, less work for the converter you'll just get them back. glTF declare those states (wherever they come from), in this issue we were discussing to best way to represent this declaration so that these states that you get can be retrieved/used efficiently. The implementation to set them is up to you, and whenever implementations details were discussed it was just an example not mandatory. This is the reason I was vague in my previous answer to Remi, to not mandate any implementation behavior (but still concerned that efficient implementation is possible).

[RemiArnaud]

@fabrobinet https://github.com/fabrobinet - before we can meet to discuss solutions to the problem, we need to be clear to how an engine (any engine) need to use the information stored in glTF to correctly render the model.

Once we have that established, we can look at where glTF may not provide enough information for an engine to be able to do optimal state management.

There are many choices to how we want to deal with this issue. Maybe we can prove by enumerating all the possible cases that there is enough information overall in glTF for an engine to make optimal state management. Maybe we decide that glTF wont enable the most optimal state management in enumerated cases and document it. Or maybe we provide a fix to glTF, or an optional information, to enable for optimal state managemen.

In order to move forward, we'd need more than a vague answer, we do need to be able to provide a specific answer. It is important that the glTF specification provide all the information necessary for an implementer to have correct rendering.

Question: starting with default state, with the following information in gltf, what is the optimal state management for those two materials/draw?

stateA: default=false stateB: default=false MaterialA: states={stateA:true} MaterialB: states={stateB:true}

On Mon, Apr 14, 2014 at 9:24 PM, Fabrice Robinet notifications@github.comwrote:

OK I see, then in your format you have gl states set by artists, just like glTF does. Great ! These would be just preserved and serialized in glTF I don't see the problem. How to land this info from the tool to glTF could be done in different way but it is another topic, just focusing on the format, if you provide states it's great, less work for the converter you'll just get them back. glTF declare those states (wherever they come from), in this issue we were discussing to best way to represent this declaration so that these states that you get can be retrieved/used efficiently. The implementation to set them is up to you, and whenever implementations details were discussed it was just an example not mandatory. This is the reason I was vague in my previous answer to Remi, to not mandate any implementation behavior (but still concerned that efficient implementation is possible).

— Reply to this email directly or view it on GitHubhttps://github.com/KhronosGroup/glTF/issues/249#issuecomment-40444425 .

[tparisi]

This thread made my head spin. I will not claim that I was able to follow the whole thing. But that's never stopped me from rendering an opinion before, so: what I can say is, +1 @pjcozzi - we need to keep it simple, and I think the current approach is simple. It's only supplied if it's not the default value. Outside of that, it's up to implementations to optimize.

[fabrobinet]

That's what I have been saying as well, no ?

Anyway, I will send a more detailed answer with an implementation proposal as asked by @RemiArnaud even though I have hold that because I didn't want that to confuse watchers of this project and have people think they had to do the implementation in a arbitrary way.

[pjcozzi]

@fabrobinet can you update this based on your offline discussion with @RemiArnaud?

[fabrobinet]

The main agreement about the discussion I had F2F with @RemiArnaud was:

• The spec should provide details about settings states management, like examples strategies to manage states to achieve expected rendering(See below for a first draft). Also provide details how texture states.
• Be explicit that for every pass/states to be processed, it is mandatory to have the GL state machine reset to its default value. To achieve this, after setting states for a given pass, some states may have to be reset to their default value. Remi propose an optimization, a “don’t care” list of states that do not need to be cleaned. We agreed that since it is not absolutely necessary for V1.0, we can wait to include this, and first step as an extension if we feel like it is important.

Example in some pseudo code, just to convey the idea… setGLStates and resetGLState can be implemented in different ways. (By either directly setting GL values, or shadow GL states…) Implementations details can varies here, the goal is just to show an example of management state using states as described in glTF.

Naive:

List statesList;

//Following is for all geometry to be drawn...

//set states and keep track in a list
pass.states.ForEach(state) {
  setGLState(state)
  statesList.push(state)
}

renderGeometry

//cleanup
states.List.forEach(entry) {
  resetGLState(entry);
}

A bit better by avoiding to unnecessary reset state that will be set anyway by the coming pass

//cleanup
Map statesMap;

//Following is for all geometry to be drawn...

pass.states.ForEach(state) {
  if (statesMap[state] != nil) {
    resetGLState(state);
  }
}

//set states
pass.states.ForEach(state) {
  setGLState(state)
  statesList[state] = state;
}

renderGeometry

//once all Geometry are rendered, do not forget to cleanup states.

Having the “don’t care” list would allow to add another condition to reset to the default state:

//in the cleanup phase like this:
pass.states.ForEach(state) {
  if  ((doNotCare[state] == nil) && (statesMap[state] != nil) {
    resetGLState(state);
  }
}

[pjcozzi]

Sounds good. So this is resolved now for 1.0?

[fabrobinet]

Yes. And kept open until spec is updated.

[pjcozzi]

I'll aim to update the schema late this week.

[fabrobinet]

No worries, I don't think this impacts the schema, at this point it is more about guidance for adopters.

[pjcozzi]

Closing as duplicate with #85.