High level tasks

[mikolalysenko]

Here is a list of some high level things that need to get done regarding gl-modules:

Community building

In order for the project to be successful and visible, we need to build up some sort of web presence beyond github/npm/irc. I think getting a slick webpage would go a long way to getting people on board and building up the community.

• Website (currently have modules.gl, but page is still messy.)
• Logo (some kind of animal? zebra maybe?)
• Demos (already have a couple sitting around, but need to gather more)
• Tutorials/workshops (already have shader-school, but need more)
• Chatroom (currently #glslify #stackgl)

  Technical

At a high level, gl-modules could be grouped into the following collection of projects:

• Modular shader programming: glslify
• WebGL wrapping: gl-buffer, gl-vao, gl-texture2d, gl-fbo)
• Initialization, input handling: gl-now
• Math libraries: gl-matrix fork
• Camera controls: game-shell-orbit-camera (this needs serious attention)
• 3D Plotting/visualization
• File parsers

Each of these projects needs some work. Here are a few broad issues though that I think we need to work toward addressing:

glslify

This project is progressing nicely, thanks to @chrisdickinson and @hughsk's continued efforts. Feature-wise, I think it is ready to go, but it could still use a bit more work regarding error reporting and user-friendliness.

WebGL wrappers

These are mostly complete, though I do want to revisit the decision to use [rows,columns] instead of [width,height] for specifying shapes. This will probably be a breaking change, but I think it is for the best long term.

There are a few odd ball missing features though, that we should look into wrapping up for simplicity:

• cube maps: This would probably be handled using a custom FBO/texture module
• pixel readout: We could probably work out something that reads out the pixels from the screen into an ndarray. Should not be hard, but could be really convenient
• general API auditing/cruft removal: Need to make sure that everything is as concise as possible. There are probably a few places where the syntax can be optimized here and there before we look into a new major version.

  gl-now

gl-now and game-shell are both pretty messy and need to be smashed into tinier modules. Ideally the input handling needs to be factored out. I want to eventually replace the input system with something that does persistent inputs to avoid some of the clock dependent issues. This should also tie into mouse and gamepad input handling so that we can do something which is a bit more unified. Ideally these modules should be something that you can just require if you want to get started right away.

Math libraries

@hughsk is already working on this, and I think it is great. Eventually though we should work toward moving away from gl-matrix and to a more modular/tiny collection of focused matrix libraries.

Cameras

Improvements here are highly contingent upon improvements in gl-now and the matrix math front. But there is a lot of opportunity here to do better. I think that the goal should be to create some small modules which are sufficient to generate whatever type of camera you like, as well as some default "uber" camera that can be configured to do whatever.

The general camera interface would be some object with a number of properties, like model, view, projection, their inverses and so on. These could be computed by the camera object and cached whenever something changes to avoid recalculating all these different matrices. Ideally the camera could be attached to the gl-now object and recompute these matrices whenever there is a render event.

3D Plotting

Finally, I have been working on some modules for doing basic 3D plotting. Once we make improvements to the rest of the core infrastructure in gl-modules, these could be added to this project and exposed as a simple way to get basic 3D visualizations working in the browser.

File parsers

There are a number of file parsers for 3D formats floating around on npm, but it might be good to try collecting them together and maybe looking at exposing some unified output format (probably a cell complex) from these.

[hughsk]

:+1::+1::+1:

Sounds great. I started playing around with a general site structure the other day, as it turns out we've already got a whole bunch of stuff to include! I tried getting that heightmap self-shadowing stuff working but got totally stuck – they're also using CUDA in their implementation. Would you be willing to give it a shot and then I'll work on the visuals? Otherwise, I have a couple of other simpler ideas that could work.

Regarding reading out pixels, I've got canvas-pixels which works with 2d and webgl contexts – in the latter case it'll read it out from the current framebuffer. Definitely agree with improving the camera situation too, simply to reduce the amount of copy/paste involved in getting a project set up.

One thing I've been contemplating: a gl-boilerplate repo for getting started quickly and easily, potentially including the nice extras such as testing, optimisation and live reloading. Also happy moving those repos over to gl-modules if you guys approve :) Similarly, maybe a standard HTML file for demos like three.js seems to have?

Some simple demos/examples of mine from a while back, might revisit them to use the new versions of everything:

• https://github.com/hughsk/indexed-geometry-demo
• https://github.com/hughsk/particle-excess-demo
• https://github.com/hughsk/audio-poly-terrain-demo
• https://github.com/hughsk/particle-excess-demo
• https://github.com/hughsk/poly-terrain-demo
• https://github.com/hughsk/web-audio-analyser

Another thing to add to the list would be those shader lighting modules you mentioned at nodeconf, they'd be super useful!

[mikolalysenko]

I think I would like to resist the idea of creating a boilerplate repo. Ideally it should be simple enough to get a project up and running that there is no need for boilerplate/copy-paste. Still it is helpful in highlighting the things that we should work on improving.

Regarding the heightfield lighting stuff, it probably isn't too important to get it finished now (unless you are really excited about it and want to make it happen!) The bigger picture issues here are figuring out how to make this stuff accessible and attractive for newbies to pick up. I am going to nodeconfeu in September, and will try to talk about this project a bit out there.

[mattdesl]

:+1:

Ideally I'd like to utilize a few gl-modules underneath some parts of kami. My main concern right now is bloat.

For example, just to render a textured triangle on screen, your minified bundle is going to be over 140 kb, already a third of the size of ThreeJS. And you're still not really any closer to making an actual application.

I've also noticed gl-modules seems to be heading in a pretty 'academic' direction, per se. Not that it's necessarily a bad thing, but I think too much of it upfront may deter some newbies. Stuff like [rows, columns], ndarray ops, and the focus on filling buffers manually. The solve for that is to produce higher-level modules and demos on top of gl-modules (be it like kami or something else entirely).

If you want gl-modules to be more flashy and newb-friendly, here are my suggestions:

• make the logo animated and in webgl. anything with wireframes, delaunay triangulation and simplex noise tends to impress people
• a tiny photo transition demo, using some shaders from glsl.io
• more practical and "production" applications of gl-modules. Stuff like handling retina canvas scaling, preloading images, drawing many quads at once, basic line rendering, etc.

In my experience, most people who are looking to learn GL either do it for games, fancy graphics, or artsy data/audio-viz.

[hughsk]

@mattdesl that's exactly what I've been using it for :) All valid points. I've gone ahead and pushed the WIP site, which I think is a little more flashy and newb-friendly – take a look. (oh, currently doesn't work in Firefox, heh.)

I'm certainly interested in making the project more accessible too. I've written a few higher-level modules such as gl-geometry and mesh-combine, but am always interested in more.

Any bloat is something we're definitely keen to address. Tools like disc are useful for tracking down what's actually causing the issues and @mikolalysenko has been working on transforms for heavier dependencies such as cwise. Last time I had an issue was with south.im, but it's improved a lot since then :)

I gave your example a shot using a fresh install, ~200kB unminified and ~100kB minified using uglifyjs -cm. The biggest contributor there is a-big-triangle which weighs in at 35kB of that. That's because gl-buffer and in turn its ndarray dependencies are taking up (relatively) a lot of space. In practice though, these modules can all be deduped and so you're only hit with the initial overhead of under 200kB. The http://nodesource.com website, for example, has a bundle of 178kB to do its stuff which is less than half of what you need to get three.js to draw anything!

[mattdesl]

Nice AO you got there. :ok_hand:

The bloat isn't really that bad, but it deters me from depending on any of these modules as a base for kami, which is (for now) primarily meant to render 2D sprites. A similar program with kami is compressed to 25kb with -cm flags.

I personally don't think the base gl utilities (buffer, texture2d, etc) should be tightly coupled with ndarray. In most production applications the images and meshes are loaded from cache rather than bundled into JS source, and they don't go through very much processing before being piped to glTexImage2D or glBufferData. For images, it's a waste of memory and JS processing to turn the Image into pixels before sending it to glTexImage2D.

[mikolalysenko]

Hi @mattdesl and thanks for all the great feedback! You should totally join us in the #stackgl IRC channel on freenode which we are starting up.

Regarding file size, I think there is a lot of low hanging fruit which we can attack to shrink the bundle size down substantially. So far I haven't really prioritized size optimizations, since a few hundred kb doesn't have any appreciable effect on performance or loading times for a single bundle -- but I will admit that file size makes a huge difference with respect to marketability and public perception (which are very important to drive adoption!). The biggest culprit in that demo I think is that a-big-triangle has horribly out of date dependencies and could probably be simplified substantially. Changing the function names in ndarray and using other simple tricks, I bet we could get the core module down to around 4k or less. It should also be possible to split ndarray-ops into tinier pieces, which can probably free up another 10k or so. There is also plenty of room to squeeze more space out of cwise & co using various tricks.

Regarding using ndarray for uploading data, I think it is really one of the main advantages of these modules over other approaches. While you can still upload directly using image/canvas/video elements, the ndarray interface greatly simplifies the process uploading binary data. In the immediate term, this is useful if you want to generate images or other data procedurally, and longer term I think it will be important when building out node.js WebGL applications (for either testing or server-side graphics purposes).

Without ndarrays, I think that modules like texture2d would suffer enormously. The main issue is that there are just too many different image formats and memory layouts to try to pass them through a single function (which is partly why the base WebGL interface for textures is so complicated). Using ndarrays makes it much more streamlined and simpler to specify things like texture orders (ie you can trivially flip/transpose images or handle RGB/BGR inconsistencies).

Using ndarrays also makes a lot of sense for buffers too. In addition to being consistent with gl-texture2d, many times you can end up with vertex data sources that are packed in some funny format, which you might want to slice into pieces for the sake of buffering. For example, perhaps you only need the xyz components of some binary model's vertex data, while the full file has a bunch of extra normal and tangent space info.

There are also longer term issue here as well. Eventually I hope to add more options for GPGPU programming using ndarrays (which should become more feasible with WebGL 2.0 and WebGL access from WebWorkers), and having interfaces that can produce/consume them directly should greatly help toward this end. I also see one of the big applications of gl-modules eventually being data visualization, and for higher dimensional data sources I think that ndarrays are a good representation for many problems.