investigate replacing canvas with WebGL

[mykmelez]

We should investigate replacing canvas with WebGL, which @nickdesaulniers and Martin Best have suggested would improve graphics perf. significantly. Martin mentioned that @juj may be able to provide more info.

[nickdesaulniers]

Libraries like pixi.js have shown that significant more bunnies can be drawn on screen with WebGL. I would bet money that simply dropping in webgl-2d would give you performance wins. Of course, custom webgl logic would give fine grain control and absolute best performance.

Before going through with these changes, you should be able to show that for a profile with an inverted call stack, most time is spent in the canvas calls (as opposed to your update logic). If the time spent within CanvasRenderingContext2D instance methods in total is less than 10%, then the switch to webgl is probably not worth it.

Having the profiles before hand would help measure performance wins.

[marco-c]

The two most used operations are drawing strings and performing pixel operations, so I'm not sure this would help. I've been trying to use the canvas debugger to get a trace and validate that these are actually the most used operations, but I can't use it (https://bugzilla.mozilla.org/show_bug.cgi?id=1122766)

[nickdesaulniers]

Rather than using the canvas debugger which will just show you the CanvasRenderingContext2D calls relative to each other, you should just use the sampling profiler to see how expensive the context calls are relative to other places in the code. If the content calls make up very little time spent in a profile, then they are not worth optimizing.

[marco-c]

Looks like the graphics-related native that takes more time is the one that decodes images ("createImmutableImageDecodeImage"), with 2,7% of the total cost.

The second is "drawString" with 1,69%.

I took a profile on desktop, because the remote profiler with a Flame isn't working for me (https://bugzilla.mozilla.org/show_bug.cgi?id=1122784).

P.S.: I took a profile using mbx's branch

[nickdesaulniers]

Eh, 4.39% of time is not really worth optimizing for, even if those were your two heaviest calls overall (not just gfx). Even if they were made 100x faster by webgl, that would only make your program 4.3% faster (check my math: 1-1/(100/(1+(1-0.0439)*(100-1))) http://en.wikipedia.org/wiki/Amdahl%27s_law#Speedup_in_a_sequential_program). Unless you were just missing your frame time of 16.66ms then it's not worth the effort, or unless it's the last thing you can do and dropping in a lib like webgl-2d is simple.

[nickdesaulniers]

Also, writing text using GL is a pretty miserable experience relative to canvas 2d, but maybe @juj has different experiences.

[mykmelez]

I'm suspicious of these measurements and would not give up on optimizing our gfx code because of it (although whether WebGL is worth it is a separate question).

[juj]

Oh, I answered to another thread before seeing the results here.

Like Nick mentions, that is a very small portion of the total execution. If you are suspicious of the timings, I would recommend either doing manual intrusive performance.now()-based profiling, or no-opping out the operations you suspect are heavy and comparing the performance increase to a baseline version. What was the biggest hotspot in the profile you took when it was not graphics?

About text rendering with WebGL: that's a more involved process, because there are multiple parts to it: (you probably are familiar with this already, but recapping here)

• one needs to be able to load fonts and convert the vector font data to a raster form. In desktop world, Freetype 2 is by far the most popular solution. It could be Emscriptenized, but also in browser side one can cheat by drawing and extracting individual characters from an offscreen canvas2d to piggyback on the browser's capability to load and rasterize fonts.
• the rasterized glyphs will need to be stored in an atlas cache, which requires performing 2d bin packing, see resources for that here: http://clb.demon.fi/
• then, one needs to implement text layout manually, i.e. aligning, kerning, spacing, glyph measuring and so on.
• actual rendering of the glyphs amounts to drawing 2d sprite quads consisting of two triangles each, where the UV coordinates pick the correct character from the font glyph atlas.

All the individual steps are "well solved", but it's a bit laborous to develop all the code to do it, and it looks like https://github.com/gameclosure/webgl-2d does not implement this at all. I'm not sure if it's sensible to attempt this, before first optimizing the existing canvas2d implementation and concluding that the performance is not adequate.

[marco-c]

What was the biggest hotspot in the profile you took when it was not graphics?

The biggest hotspot is the Java interpreter.

The most used gfx operations are: stringWidth drawString getRGB (this reads pixels from a canvas) drawRGB (this draws pixels on a canvas) drawRegion (this draws a canvas on another canvas)

I've tested a few things with performance.now()-based profiling, they're negligible compared to the interpreter. After we've enabled JIT compilation, when the interpreter cost will be reduced, these might become more interesting (for example, 2% might become higher).

First of all, I'll try to refactor the gfx code not to use closures. After this optimization and #874 we will have a better sense of how much performance we can squeeze out of gfx.

[marco-c]

897 is removing the callback arguments from the with* functions.

[andreasgal]

How hot is getRGB? Reading back textures is a huge pain. drawString we can make a lot faster using WebGL if we use a texture atlas for glyphs (it becomes essentially free).

On Jan 21, 2015, at 3:29 AM, Marco notifications@github.com wrote:

What was the biggest hotspot in the profile you took when it was not graphics?

The biggest hotspot is the Java interpreter.

The most used gfx operations are: stringWidth drawString getRGB (this reads pixels from a canvas) drawRGB (this draws pixels on a canvas) drawRegion (this draws a canvas on another canvas)

I've tested a few things with performance.now()-based profiling, they're negligible compared to the interpreter. After we've enabled JIT compilation, when the interpreter cost will be reduced, these might become more interesting (for example, 2% might become higher).

First of all, I'll try to refactor the gfx code not to use closures. After this optimization and #874 https://github.com/andreasgal/j2me.js/issues/874 we will have a better sense of how much performance we can squeeze out of gfx.

— Reply to this email directly or view it on GitHub https://github.com/andreasgal/j2me.js/issues/880#issuecomment-70822234.