Kode / Kha

Ultra-portable, high performance, open source multimedia framework.
http://kha.tech
zlib License
1.49k stars 169 forks source link

instances of g2 are not independant while drawing. #740

Open nanjizal opened 6 years ago

nanjizal commented 6 years ago

instances of g2 ( perhaps g4 ) are not independent ( webgl ). for instance drawing within one can have unpredictable results. http://kodegarden.org/#2988b4eb95078f0cec765cb0e1216956be4c746b http://kodegarden.org/#9c9c361f496b70acb912b38844c52ede4235cef5 please confirm if this is a limitation of Kha/Webgl as it's implications make more complex render structures significantly more fragile and complex, and it's a side effect not normally expected, ie totally at odds with functional styles of code. Understand if it's a limitation of the tech and keeping it fast, but felt that I should raise the issue so that documentation could be considered to explain this for other users, especially as the render results can be unpredictable and so cause more confusion.

Disar commented 6 years ago

I'm having a bit of trouble why you need it to be that way. You always call end before calling a new begin.

I do know that begin binds the render target and restarts the process for g2 where the vbo starts again at index 0. end needs to be called to flush to the image buffer.

when you call drawXYZ functions ( i.e. drawImage(...) ) you don't actually draw anything, what g2 does is place items in the vbo. It adds the vertex position, uv and color data. When either a state changes: i.e a texture or a shader the vbo is forced to be flushed and drawn to the image buffer. When the vbo is full g2 also draws the buffer and starts at 0 again. But in any other case you must always call end or the vbo is never drawn to the image buffer.

I might have got something wrong, if so please correct me.

nanjizal commented 6 years ago

Disar the type of code might be something like:

g2.begin();
g2.clear( Color.White );
for( myComponent in scene ) myComponent.render( g2 );
g2.end();

Now in some of your components you draw textfields that have non standard spacing so you draw each letter, or you want to use something like polyK to draw complex vectors from lots of triangles, but you don't want to render them lots of times even if the x and y of them is translated. So you cache the render result for them and you might have some code like this:

function render( g2 ){
if( imageTri == null ) {
imageTri = Image.createRenderTarget( Math.ceil( dim.width ), Math.ceil( dim.height ), TextureFormat.RGBA32, DepthStencilFormat.DepthOnly, 4 );
imageTri.begin();
imageTri.color = Color.red;
for( tri in triangles ) imageTri.fillTriangles( tri.ax, tri,ay, tri.bx, tri.by + tri.cx, tri.cy );
imageTri.end();
} 
g2.drawImage( imageTri, x, y );
}

Each component may have different x and y that you maybe tweening. Now you can argue that you can pre-generate the image, but only if you realize you can't do this. So instead you end up with more complex code.

for( myComponent in scene ) if( myComponent.needsPreRender() ) myComponent.preRender();
g2.begin();
g2.clear( Color.White );
for( myComponent in scene ) myComponent.render( g2 );
g2.end();

If you start implementing really complex structures like scrolling virtual lists with dynamic content in each item that could change then your going to need this structure replicated throughout the render chain. Now start doing dynamic layout based on how big content is rendered, like flowing text round a images, now your code is for prerendering has to be done after the render code but maybe it's based on aspects of the normal rendered code. For instance you want to treat complex text fields more like they are just g2.drawText( commands, so complexText.drawText( g2, str, x, y, renderAsImage );

So you always end just you may want to create images to use within the begin and end. To use g2 do I need to know about vbo should I care, I just want to use it like some normal draw commands, imagine canvas you can create other canvas within the drawing loop and then draw them to the canvas. For instance I use two Canvas to draw zebras probably within a loop, so one canvas start draw process, second canvas draw image rotated and then draw it to the main canvas, till all the limbs are generated and then end the main canvas draw loop. https://rawgit.com/nanjizal/Zeal/master/deploy/index.html So this uses a 'leaf' system to help setup the offset rotations and nesting. https://github.com/nanjizal/htmlHelper/blob/master/htmlHelper/canvas/Leaf.hx I am not having to be careful each context is isolated and I can draw to it within the draw process of another context drawing. https://github.com/nanjizal/Zeal/blob/master/src/zeal/Zebra.hx#L104 So to have a Zeal ( group of zebras ), the draw process might be a bit like... 1) start main canvas drawing. 2) draw Zebras 2a) draw rotated Limbs using another Canvas 2b) draw them on main canvas 3) end main canvas drawing. I am probably only using 2 Canvas but I don't create lots of images I reuse one canvas lots of times and draw it to the main canvas during the draw loop, how would I do a similar process in Kha say I wanted to create one image and reuse it within the main draw loop perhaps. The g2 all being linked for me is confusing I don't know about vbo unless I have used it playing with g4, but I do know about drawing graphics in flash, js and swing and I don't remember having to becareful about ending before starting. Is my reasoning more clear? It seems natural to me to want to nest g2 render operations using different images and with Kha at moment that seems to break.

Disar commented 6 years ago

To use g2 do I need to know about vbo should I care,

I'm simply pointing out how g2 functions as it's very common for a "batching" system to work like that. g2 performs best if you can draw many things without changing state. Thus calling end() is important in this particular case. It's simply how Kha is designed currently, which is a very common implementation.

Creating many rendering targets ( even if they are small ) isn't going to be any more efficient, and imo if not handled with care can actually cripple g2/performance due to constant render target binding. Redrawing the whole screen is probably still more efficient ( but you'd really have to bench mark this ).

2a) draw rotated Limbs using another Canvas

Why? Nesting and transformation can be done with a single Canvas. Your problem is in high level structures and how you control the flow of your rendering.

, but I do know about drawing graphics in flash, js and swing and I don't remember having to becareful about ending before starting. Is my reasoning more clear?

I have a feeling you're conflating render targets with display containers. Canvas in kha is not the same thing as Canvas in JS. Canvases are Images and Render Targets. Not hierarchical concepts.
Display containers are high level concept that simply hold drawing information and child relations. It's what I currently do in my project as well. And you can build something similar. If you're working with OpenGL or DirectX directly you will run into similar problems, it's low level and leaves the flow of control over to the user in higher level concepts. In my current project I have a simple DisplayObject concept, begin g2 and let it run until the end of my render loop and then call end. So my loop looks something like:

update:

render:

I don't touch the end() function in any complicated way even if I change shaders or textures. Every display object handles it's own transformation and child relation. If one of the components is using g4 I make sure I end g2 first and continue with g4 and vice versa. My display objects simply holds rendering information and a render function so I can control how the display object renders itself. Note that this is a fairly naive implementation due to how I mix g2/g4, but it works for me and is efficient enough for what I'm trying to do. If you're just doing g2 this probably is fast enough.

Take OpenFL for example, it runs on Lime which is driven by OpenGL. Yet it mimics flash exactly the same with its display objects. When you work with the OpenFL API you can stack/parent/transform display objects without touching OpenGL, but it is most definitely running on OpenGL. Don't forget that the begin/end concepts are used in Flash as well for vector drawing by code, not for bitmap data ( which is done in the background trough their tiling api? )

If you're doing UI work it's perhaps handy to only redraw the regions your mouse is hovering over. ZUI does this and doesn't use additional render targets.

To achieve what you want you have to create a higher level structure like a scene/graph renderer. It's probably best to look at some open source projects how they handle their rendering pipeline, Armory3D is also a good contender. Learning about linear transformations/matrices is also a must for parent child transformations.

That said, kha is fairly low level and doesn't assume much about how things are implemented from the users end. It could probably use some additional features. But Khas intend was never to provide the same high level concept you see in flash. It's up to the user to create the system they wants/need.

RobDangerous commented 6 years ago

Stop being silly, dudes.

RobDangerous commented 6 years ago

Here, I fixed your code: http://kodegarden.org/#9412ccdf9ab4531881db026a00311f45c15de6b7 Please don't create a new render target each frame, that might blow up some computers. More importantly though you have to end before you begin, always. This way it works fine via the html5 target but there's indeed an issue in KodeGarden aka the html5worker target - will keep this open to fix it.

Disar commented 6 years ago

Im hoping the creating a render target every frame was an oversight ( I noticed it too) . But I don't think that's Nanjizals issue. I think he is struggling with incorporating kha with some higher level concepts. It isn't so much a kha issue I think as it's a architectural design issue. Im definitely sure Nanjizal wants something akin to Flash's displaylist. Again Canvas ( images/render targets) are not hierarchical concepts in kha.

RobDangerous commented 6 years ago

I don't care, I just want to fix bugs :-P

Disar commented 6 years ago

Yes I understand, I'm just pointing out his problem. Close it at your own will ( or wait until nanjizal does if he has something to add ).

nanjizal commented 6 years ago

Robert thanks your example make sense, thanks for making the time to illustrate the concept. I am still wondering, there is no way to know if a g2 or g4 process is underway or accessing the process, so the user might need to pass around the active g2, how would this work with threads, I guess outside java haxe is not doing so much with threads, but I am presuming it's still a limitation? If you begin end lots of times is it likely to be heavy, or little difference?
Out of curiosity would providing access be expensive or even desirable.

if( Graphics.began() ) { 
var graphicsState = Graphics.current.state();// opacity, color ..
Graphics.current.end();
}

Sorry my limited understand is likely annoying, but I doubt I will be the last! Feel free to close, but would be good to add something to documentation perhaps with Roberts example above with maybe some below the scenes details, happy to help with creating diagrams, if you let me know what's needed. Feel free to close you have confirmed it's a feature rather than bug.

nanjizal commented 6 years ago

Oh another question if you createRenderTarget say in the update code could it happen between g2.begin/end ?

Disar commented 6 years ago

Render targets can be created outside of the render loop. It only creates an image buffer you can bind and write too. You should just create render targets where you initialize stuff and swap between those.

But again I wouldn't use many render targets, especially on devices with low memory.

RobDangerous commented 6 years ago

g4 and earlier is single-threaded, just like OpenGL or Direct3D <= 11. begin/end is heavy and should always be used with care. createRenderTarget can go anywhere AFAIK.

RblSb commented 6 years ago

Since this is a very difficult thing to debug and is so hard to find in the documentation, I suggest making the counter of open "drawing instances" and throwing an error if there is more than one open when debug flag is used.

RobDangerous commented 6 years ago

Aye, I will do that.