Feature Discussion: gl-matrix 2.0

[toji]

So I've got a bit of free time on my hands and wanted to direct it at something I've been meaning to do for a while: Start the second iteration of the glMatrix library. This issue is to collect feedback on the proposed changes before we go crazy implementing them.

To be clear, the bump to a 2.0 library would indicate breaking of backwards compatibility, and thus several ill considered design decisions/flaws can be corrected without concern about breaking existing usage. The library in it's current form will likely stick around for a while longer and receive a few minor bug fixes, but new features would be 2.0 facing.

A few items that I feel must be part of the refactor:

• The current method of using array-like objects will stick around. The library will continue to create typed arrays by default, but can accept anything numerically indexable.
• As proposed in #41, the semantics for input/output parameters should completely standardized. ALL functions would require an output object where applicable, no more implied change to the first/second param.
• Related to the previous item: Parameter names should be consistent. Names like "vec2" are confusing, since vec2 is also a "type" in gl-matrix. Preferred naming conventions for parameters are "a", "b", and "out", unless a different name would clarify the usage substantially.
• Standardize error handling. There are a couple of cases (matrix inversion, for example) that may have no valid solution. Currently the error handling is "return null and hope they notice." but that's not terribly well defined. Whatever method is used to handle errors, it needs to be well documented and consistent.

In addition there are some things that I would like to see

• I've long lamented the fact that the symantics for matrix multiplication are essentially "backwards" in gl-matrix, as noted by this Google code bug. I feel that any applicable operations should be changed so that the result is a more natural out = a * b instead of the current out = b * a. My only reservation here is that it may make porting code slightly more confusing.
• When gl-matrix was first introduced TypedArrays were fairly new. Now even browsers without WebGL support them, so we should investigate ways to utilize them better. A specific example would be creating a "clone" function that passes the structure to copy into a Float32Array constructor.

Finally, there's a couple of items that have been proposed by the community but that I'm not sold on:

• This Google code bug suggests changing to a non-namespaced pattern, so that mat4.multiply() would be mat4Multiply(). The reason given is that it would be faster (and it would be, if only a little!) but I'm not sure if I like the idea or not.
• I've heard a couple of suggestions that along with requiring an output parameter on all operations we also introduce a set of functions that explicitly alter the first param, most likely named ___Self. a.k.a: vec3.translateSelf(a, [1, 2, 3]); would alter a. This would allow us to retain some current functionality and may even be faster is some cases, but the API is pretty large as-is, and I'm a little reluctant to clutter it further.
• Stop returning the output parameter. This would (theoretically) be faster, but would eliminate the ability to nest functions.

I'm sure there's other items I can add to this list, but that's what I've got off the top of my head. More feedback is appreciated!

[ioquatix]

I haven't used glm but I have looked at it briefly.

You should be using column major multiplication which is standard for OpenGL, this means that a * b * c * vec is read from right to left, e.g. a(b(c(v)))) - e.g. apply c to v, then apply b to that result, then apply a to that result. This is natural for mathematicians and pretty much the defacto form.

Using typed arrays makes sense and hopefully allows some of the operations to be optimised.

If people have a problem with having a prefix, why don't they just write var m = mat4.multiply and call m(...).

vec3.translateSelf seems like a bad idea. If people want to optimise the code to minimise copying you could have a separate module for doing that.

I'd be inclined to put all the code in one namespace e.g. glm.mat44.multiply

[ioquatix]

Another thing you might want to consider is applying a simple multiply to multiple vertices, e.g.

glm.mat44.apply(m44, vertices)

This allows you to provide efficient implementations of matrix multiplication and avoid temporaries.

The same can be done for chaining multiple multiplications, e.g.

glm.mat44.product([mat1, mat2, mat3, mat4, mat5])

By doing this in one function we can avoid making 4 temporary buffers.

It is pretty common to see this use case e.g. when making cameras, traversing a scene graph, etc.

[jagenjo]

Great work, I think there is no need to ditch the namespace to improve performance, if anyone want it it is easy to create bindings.

Here is a list of methods I missed when using glMatrix: vec3.toArray: to non-typed array, I need this when using stringify. vec3.min: given two vectors returns a vector where every value is the min of both vectors vec3.max: the same but max value mat4.rotateVec3: as converting to mat3 and multiplying the vector but direct quat4.toEuler transform the rotation in 3 simple rotation along the basic axis (its useful when you want to show the rotation in a user friendly and manipulable way). quat4.fromEuler: the oposite.

And I also miss a lot functions to create matrices with a transformation, like setTranslation or setRotation instead of having to rotate an identity matrix, that annoys me .

Thanks again for you amazing effort. :)

[kpreid]

I like the idea of the ...Self methods. I would use a single-letter suffix (us GL programmers are used to it, and there's some good old precedent (to me) in, for example, Common Lisp's n... functions). For this library, I don't mind having a large API — the job of gl-matrix is to provide exactly the operation I need in the most efficient form. However, it would be nice — even for the current version — if there was a ‘quick reference card’ style documentation page which listed each function and the roles of its arguments (including left vs right multiplication) in an extremely compact layout — one line per function, or nearly so.

Please don't un-namespace the methods; if we want speed we can always explicitly assign them to local variables (which in future JS versions will likely have nice syntax as a "module import" facility), and my understanding is that local (or even closed-over to some levels) variables are sometimes significantly faster than global variables since the lookup for global variables is hairily dynamic.

Suggestion for error handling — return a vector/matrix of NaN. This has several advantages: (1) NaNs may already occur in numerical code written by the user, so if they have the sort of code with exceptional cases they may need to check anyway. (2) The user can do a chain of numerical operations and check for NaN at the end, rather than having to test for a null matrix at every point gl-matrix might return one. (3) If the user doesn't check at all and has a rarely occurring failure case they didn't think about, then they'll just see some disappearing geometry or similar occasionally, rather than possibly crashing.

[sinisterchipmunk]

Disclaimer: there are an infinite ways to read the English language. The following are all merely my opinions based on my own experiences over time, and shouldn't be taken as anything but. If anything implies an accusation, it's not meant to. Also, apologies for the length. I had one thought, and then I had a lot of them.

• I'm unconvinced but largely uncaring about the Self methods. I know right away that I likely won't use them; on the other hand, that doesn't mean they're not useful and maybe even more efficient (though I'd like to see this benchmarked). I just find the construct a bit ugly. I love the terseness of gl-matrix: mat4.multiply, vec3.add, and friends all give a massive amount of information about what they do without adding the overhead of extra typing (as compared, for an extreme example, with something like Matrix4x4F.multiplyMatrix4x4F). Adding Self to the end of any given method makes it less terse, confuses the meaning just a tiny bit (in my own mind at least), and adds at least 5 keypresses (including SHIFT) to any given invocation -- not to mention leading to tired pinkies. :( Again, not saying they won't be useful, but if it was up to me and only me, I'd probably skip them in favor of less a bulky API.
• One more note about the *Self methods. I don't like having too many ways to do any one thing; it makes the API at large more confusing and steepens the learning curve. If an operation can be performed another way, IMHO we should be scrutinizing the value the extra functions are actually adding.
• I'm with @kpreid on the namespacing 100% -- it was my first thought on the matter. Namespacing is your friend, and not namespacing is bad practice in JS, asking for collisions at one point or another. If it weren't so wordy, I'd be in favor of GLM.mat4.multiply, even. I'm not in favor of something like GLM.mat4Multiply, though. To me, something like that loses readability. I love the current namespacing. Let's keep it.
• I also love @ioquatix's suggestion about applying transformations to a set of vertices (at least I think that's what he said). Something like mat4.multiplyVec3s(vecs), where vecs is a 1D array. I would stay away from nested arrays or variable arguments though, as these are both slow. I think the idea of combining multiple operations is novel, but I don't see how it can be done efficiently without introducing the unnecessary creation of temporaries (as shown in ioquatix's own example). I think such a thing should be thoroughly benchmarked before committed to; it may be better to simply encourage reuse of cached variables even if it's a little more wordy.
• Operation order, IMHO, should be identical to what we (as GL coders) see in the shaders. mat4.multiply(mvp, p, mv) should be equivalent to the GLSL code mvp = p * m. We need consistency not only in our own APIs, but across the realm of APIs that are likely to be worked with. Same goes for column ordering, as it's crucial to be able to toss gl-matrix objects directly into a shader's matrix uniforms and have them Just Work (TM).
• I don't think there's any beautiful way to handle error conditions. The most correct would be to throw an actual Error. That'd also be the most difficult for developers to work around, having to wrap basically every potentially-offending operation in try...catch. IMHO, that's not acceptable. I'm not opposed to returning a matrix of NaN, but honestly in my own code an error condition has always been due to invalid methodology on my part. In these cases, I've been perfectly happy with them returning null, as this gets exposed in my JavaScript console immediately as a catastrophic error with a stack trace -- exactly what I've wanted in each case. I don't refute that there are edge cases, but I say let those edge cases check for null explicitly, and let the current implementation prevail. It's easier to track down the source of the error for the rest of us.
• Another note on errors. Personally, I like to see them in development as often as possible. It helps me vet the code prior to release. In production, I hook up an onError listener to intercept, report on, and possibly recover from such things. So we can throw errors or return null and let the developer decide how best to recover, if at all. If we use NaN, we take away the ability to do such things under the assumption that the developer will always know about NaN conditions prior to release.
• I'd love to see the new version of the library thoroughly test-driven, instead of tossing unit tests in as an afterthought. This not only builds our own confidence in the system, it also gives newcomers very nice examples of how to use the library, and it also sets a precedent so that each pull request can be rejected if it doesn't include a corresponding test demonstrating the need (and use cases) for the new code, especially for when we don't understand it due to complexity or language barrier. As it is, I feel bad about accepting pull requests that are untested, but I feel hypocritical rejecting them on that basis because our own testing discipline has been lacking.
• Please, let's start tagging releases. This gives us a better way to track APIs, changes and regressions over time; it allows other coders to "lock down" the version of gl-matrix that they are using; and it gives us an excuse to make changes to the master branch willy-nilly, because we can say, "Why are you linking directly to gl-matrix.js on master? We have tags for that." Master should be used for active development, and people should never link their pages directly to a git repo's master because of that -- but right now they have nowhere else to link to. (I disagree with the linking practice entirely; they should cache their own copies of gl-matrix, but that's another matter.)
• I second the cheat sheet API docs idea. Most of the names tell the developer all s/he needs to know; the full signature (name + arguments) tells the rest of the story. With more API consistency (always or never return a value, always make that value the receiver, always have a receiver as the first argument, and always do [whatever] on an error), we don't even need to document such things for each method. Just doc them once at the top of the page, and then throw up the method names and arguments. If the whole thing can be printed on a single sheet of paper, bonus.
• I like the returning of values. I'm all in favor of speed boosts, but we have to draw the line at some point: browsers should be fast at JavaScript, period, and a fundamental thing like returning a value from a function should not incur major overhead. If it does, we need to start harassing the vendors more. Even so, I could see removing the return values if benchmark shows conclusively that the performance gains are massive; but otherwise I'm of the opinion that returning the receiving object vastly simplifies the use of gl-matrix and I'd hate to see it go.
• Let's split the gl-matrix.js file into chunks. Create a lib directory, and let each file be named after its namespace (mat4.js, for example). Then aggregate them together during build. This can be added to the Rakefile very easily; then something like rake release is all you need to build, minimize, tag and push the latest version. Likewise, let's pull the bootstrapping code into its own file, and make that whole process a bit cleaner.
• Let's integrate with travis-ci. This way any time a commit gets pushed or a pull request gets merged, the test suite runs. If anything breaks, we'll know via email the moment it happens. Obviously, this goes hand-in-hand with the unit testing. We won't get this info unless we test.

[toji]

Thanks for all the great feedback! Feel free to keep it coming!

So far popular consensus seems to be settled on a few points:

• The output parameter will be the passed first. (mat4.multiply(out, a, b)) This was a surprisingly popular request!
• We'll be keeping the namespaces.
• There are several requests to provide functions that operate on an array of values. I think this is a lower priority overall, but it's a good suggestion that I think it worth pursuing once the current functionality has been refactored.
• Unless there is a MASSIVE performance gain to be had (there's probably not) returning the output parameter is a popular function and will be staying

Opinions are split on the "Self" function variants. I think we'll be leaving them out for now.

On the subject of Error handling, I think @sinisterchipmunk's analysis makes the most sense: It's usually more important to be able to track errors than to gloss over them. As such while the idea of returning an array of NaN's is somewhat appealing I think we'll probably stick with null returns instead. I do think that a "debug version" of the library is worth considering, though, that would add a lot of error checks and throw detailed messages to help track down issues. That's probably lower priority, though.

There's a lot of other suggestions that have been given, and a lot of them are very appealing. They'll be evaluated as we go. I did want to mention that specifically that I back everything @sinisterchipmunk mentioned, especially the points on test-driven development and splitting the library into multiple files during development (obviously there would be a single "production" file available for easy distribution)

[toji]

Copying suggestions from Won Chun on my Google+ post here for easy referencing:

• I strongly support having distinct 2 (mutating) and 3 (explicit destination) interfaces as opposed to if()-style overloading.
• Definitely keep the namespace.
• You have Matrix inverse, but no adjoint, which is the more useful of the two when dealing with homogeneous coordinates. Dividing by a scalar, like the determinant is a waste of time and precision, and and opportunity for numerical instability).
• Re: parameter order and such: ryg has some very clear prose on the matter: http://fgiesen.wordpress.com/2012/02/12/row-major-vs-column-major-row-vectors-vs-column-vectors/
• Global epsilon is somewhere between a hack and a lie. With floats, there are times you care about relative error (scale factors, such as matrix coefficients) or absolute error (post-transform positions).
• Bikeshedding: "equal" is usually "equals"
• I don't think quaternions should have all those branches in there to deal with the double-cover. What is a double-cover "problem" to some people is a double-cover "solution" since you can use the sign to control the interpolation direction (long v. short).
• Slerp is overkill much of the time. You should also support lerp, even though it isn't velocity-uniform. As a bonus, lerp commutes which lets you more easily compose them. I guess you could just use vec4.lerp in that case, so I guess you would just need an alias
• String representation of quat and vec4 should probably be distinct.
• Bikeshedding: Multiply -> Mul throughout. I don't think the shorthand would affect readability much.
• Array-based (e.g. transform N vec3s with this matrix) application would be awesome. Support explicit start/end bounds instead of requiring a view, since TypedArray creation is so bloody expensive.

[toji]

Starting on code now! Work on 2.0 will happen in the 2.0-dev branch until it's ready to be merged back into master.

I did vec2 first, since it's one of the simpler types and makes for a good testing ground. It'd be nice to get a bit of feedback on it before I start applying the same patterns to the rest of the library. Notable changes are:

• Unit Tests for EVERYTHING from day 1. You're welcome, @sinisterchipmunk. :)
• All vec2 functions are in their own file now. Will be combined with a build process for release
• out is always the first parameter where applicable, and is always required
• vec2.create now explicitly creates vectors initialized to [0, 0]
• vec2.clone handles the functionality previously provided by vec2.create(vecToCopy)
• vec2.createFrom is now called vec2.fromValues
• The output of vec2.str has changed from "[0, 0]" to "vec2(0, 0)" to address Won's concern about string representations being ambiguous for certain types (quat/vec4)
• Since vec2.cross requires an output vector the old alternate code path returning the z component has been removed
• The following functions have been given aliases:
  • vec2.subtract -> vec2.sub
  • vec2.multiply -> vec2.mul
  • vec2.divide -> vec2.div
  • vec2.distance -> vec2.dist
  • vec2.squaredDistance -> vec2.sqrDist
  • vec2.length -> vec2.len
  • vec2.squaredLength -> vec2.sqrLen

There's still a few missing functions, and I'm working on implementing a couple more of the requests (like min/max), but this should Any of the above changes (except for the unit tests) are up for discussion, so if you feel strongly about them one way or the other please speak up! I promise no offense will be taken if you don't agree with me! :)

[gero3]

you should provide a copy functionality too

vec2.copy = function(out,a){

  out[0] =a[0];
  out[1] =a[1];
  return out;

};

[ioquatix]

@toji I made a rake script for building minified release of a collection of JS files for another of my projects jQuery.Syntax, it would be perfect for building a minified release automatically, let me know if you are interested.

[sinisterchipmunk]

Sorry I didn't remember this one sooner, but how do we feel about CoffeeScript? It has a concise and pretty syntax, and bills itself as generating JS that runs as fast as or faster than its human-typed counterpart. In my own experience, this has proven true. I've watched the code it produces carefully because I use it for webgl, and it has even generated faster code than my own a few times.

We can hook it up to rake to automate the build process if we decide to use it. Thoughts?

[sinisterchipmunk]

Added Rake tasks for building, minifying, testing and releasing. Please see 4d6e2c81c5fb92669942772ad5a19861a15b3551 and c4c1cf8f90c45942b3ef3b95ae3e96925cb61a1d. Doing so required some directory restructuring. Hope I didn't step on anybody's toes.

I agonized for a while about whether dist/gl-matrix.js and dist/gl-matrix-min.js should be committed or ignored, and whether they belong in lib instead.

My reason for committing them under dist is that they really always represent the entire build, so they belong in dist; and that HEAD basically represents the "Edge release" (if that's the right term) so its product should be accessible to people who are living on Edge but not developing the library itself. It also adds a layer of safety, in that the release task requires all changes to be committed before it'll go ahead with the release, to prevent releasing old code. If the dist files are ignored, then they can't be validated that way and release would just tag the most recent commit (assuming no other changes were made) -- which could feasibly lead to tagging the wrong code.

Naturally, this decision is open to debate if anyone strongly disagrees with my approach.

[ioquatix]

@sinisterchipmunk For my build system, I included a YAML configuration file which includes the destination path. This way it makes it easy to deploy into a separate location. The YAML configuration file could be updated to include whether the result was minified, for example, rather than building separate minified and non-minified versions (which, technically, in my opinion isn't really fantastic).

[ioquatix]

@sinisterchipmunk I don't know the dist convention, but I am familiar with tagging versions or using a stable branch. Wouldn't either of those two approaches be more canonical?

[sinisterchipmunk]

@ioquatix the release task committed above does tag versions, but the files themselves needed to go somewhere (and we aren't publishing them to any website, to my knowledge). As you can see in the first commit, I was originally going to have them ignored during development and only re-generated at release. What made me flip-flop was the release process itself. The release task checks whether there are changed files waiting to be committed; if so, the release is aborted for fear of tagging the wrong commit (and therefore, tagging outdated/unstable code). This check seemed like a good idea to me, but only mattered if the files were committed in the first place. It condemns us to committing frequently-changing generated files, but I thought the pros outweighed the cons there. Do you disagree?

I think your YAML configuration idea is an interesting one; I hadn't thought about it from that angle. Are you suggesting that the configuration could control which type of file is generated (minified vs not), depending on what the action being taken is?

One more thought. The code under src essentially contains our non-minified base. So, we could have all code generation default to a single minified file, discarding the non-minified one entirely; even the unit tests would run against the minified version. This is a very safe option, because not only does it remove human error from the release process, but if something should go wrong with the minifier itself, we'd know as much during development, and probably right away. This change would throw away useful backtraces, but we could always leave a rake task in place for generating a non-minified version in order to reproduce issues where a backtrace is necessary.

[ioquatix]

@sinisterchipmunk While a lot of people do it, I personally try to avoid pushing compiled/minified/production versions of code back into the source code repository. I normally checkout the source code to my-website/materials/library then have a configuration file that tells it to install to my-website/public/_static/library

[toji]

@sinisterchipmunk - I like 'em in the dist folder, though I think I may need to restructure how the individual code files are done looking at the output. As for CoffeeScript, I'm typically not a big fan of languages that you can't debug in their original form. I would be interested to know if it does produce more efficient JS than what we've got now.

@gero3 - I was actually wondering about wether to call that function "set", like in the previous code base, or "copy". Copy seems to fit better with "create" and "clone" but I'm slightly reluctant to change the name just for the sake of changing it. Really I'm fine with it either way, anyone else want to cast their vote one way or another?

[toji]

@sinisterchipmunk, I can't get the bundler gem to install on Mountain Lion. Does it still work for you?

[sinisterchipmunk]

@toji Yes, just tested on an MBP that was upgraded to Mountain Lion yesterday. Worked fine with system Ruby (1.8.7).

[gero3]

This is copy for me.

vec2.copy = function(out,a){

  out[0] =a[0];
  out[1] =a[1];
  return out;

};

adn this is set for me:

vec2.set = function(out,a,b){

  out[0] = a;
  out[1] = b;
  return out;

};

[toji]

So, an interesting performance characteristic I've discovered: It seems that creating an empty Float32Array takes just as long (and is sometimes slower!) than creating one initialized from an array. You can check it out yourself with this jsPerf.

With that in mind, it seems like a sensible idea to initialize new Matricies created with mat_.create() to an identity matrix. That eliminates the need for one function and gives every matrix created a sensible default.

[sinisterchipmunk]

it seems like a sensible idea to initialize new Matricies created with mat_.create() to an identity matrix.

I prefer this approach. On my first day learning to use gl-matrix, I made the incorrect assumption that this was already the case, and lost some time trying to figure out why my matrices were wrong.

Additionally, a well-written app won't be continually creating temporary Float32Arrays. So even if it were slower, I'd still be in favor of this for its sheer intuitiveness.

[toji]

A some new code to take a peek at and few more comments.

The basic mat2 implementation is in place, and as always I'm happy to hear comments on it before we propagate the patterns set in it throughout the rest of the matrix types.

I'm a little torn on wether or not I care to implement methods like mat2.set or mat2.fromValues, because extrapolating it out I can't see anyone really wanting to type out 'mat4.set(1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1);` or similar. Maybe I'm wrong, but those seem like methods that are simply better suited to vectors.

In a related subject to the vector set methods, though, I'm wondering what the best way to proceed with a previous request would be. @jrus mentioned at some point that he would like to see vec2(), mat2(), etc. be used as an alternate way to create vectors/matrices. I like this idea, but the more I think about it the more I would prefer the syntax to be:

var v = vec3(1, 2, 3);

Effectively making the function version of each type an alias for type.createFrom(). I like this because it mimics the syntax from GLSL, and I've found that I use this sort of initialization frequently in my own code. It does somewhat clash with my earlier statement about matrix creation functions not feeling appropriate with long lists of parameters. however, and some people may find it to be less convenient than a straight up:

var v = vec3();

I'm not opposed to doing some if/then overloading on these specific functions, though, as they are intended to simply be convenience functions and you really shouldn't be creating vectors/matrices often if you can help it. So maybe we can have it both ways?

Finally, I've moved the functionality of what used to be mat2.multiplyVec2 into vec2.transform. The diving force behind that decision is that I feel pretty strongly that the functions should be grouped under the type they output, and thus having a vec2-producing function under mat2 felt awkward and broken usage patterns in the previous versions. I'm not completely sold on the name: I could also see using multiplyMat2 or transformMat2. Either of those may be more appropriate, since there's a decent use case for transforming a vec2 with a mat 3, or a vec3 with a mat4.

A few last personal responses:

@sinisterchipmunk: It's settled then! Matrices will initialize to the identity.

@won3d: Added adjoint as requested. I'm curious to lean more about it's use in graphics dev!

@gero3: I like your distinction between copy and set. I've already implemented it in vec2

[gero3]

44 should improve the speed a bit altough not tests are in it??

[toji]

In keeping the spirit of @gero3's proposal, I've added a common.js file that will hold various common utilities and aliases for faster calling.

I'm slightly concerned about the implementation of invsqrt, however. I tried to make use of it in vec2.normalize, but found that the accuracy was pretty bad. For example, attempting to normalize [5, 0] would yield [0.998448825932014, 0] instead of the expected [1, 0]. 0.0015 is a pretty big margin of error, relatively speaking, so I'm a little reluctant to push that as the standard normalize method. Of course, I know that the algorithm came from the Quake code, and there's a part of my brain that wants to say: "Good enough for Carmack == good enough for me", but I also don't want to limit the scenarios the code can be used in.

I think that the real solution is to have a simple configuration section at the top of the generated files (unminified) with basic options like:

• FAST_INVSQRT
• USE_TYPEDARRAYS
• COMPARE_EPSILON

and so on. These would mostly be options that take effect once during load. Then the user could choose their own performance/accuracy characteristics.

[toji]

Making a note to myself for future reference: I've always found the D3DX Math libraries to be extremely useful. We should seriously consider adding functionality that it provides that we lack.

http://msdn.microsoft.com/en-us/library/windows/desktop/bb172972(v=vs.85).aspx

[toji]

First pass at vec3 is in place, mostly just copying and updating the functions from vec2. The interesting functions like unproject and rotationTo are still in progress.

[toji]

Got vec4 in tonight as well. Again, these are mainly functions copied from vec2 and vec3. Still need to review and add any vec4 specific stuff.

[Juriy]

Not sure, if someone else would find it useful, still...

I often find myself doing the the pair of calls: to trim the vector. First normalize (to get the vector with the length 1), then scale by the value. This is the regular operation when you deal with speeds and steering: you shouldn't allow the speed to be more than the certain value, so you trim the vectors at some point. How about including the following two functions to the library (not taking the mutable arg/return type into the account):

• setLength(a, scalar)
• trim(a, scalar)

[toji]

@Juriy: I actually do the same in my code all the time, and had been considering adding that functionality. Thanks for the reminder!

[ioquatix]

@toji, @Juriy, the best way to support this is to have a default argument to normalize, e.g. length = 1, and allow you to call normalize(vector, 3) or something similar. The code path is efficient that way and API is minimised.

[toji]

Okay, I'm poking at this a little bit again. Sorry for the downtime: I've started a new job AND gained a newborn son within the space of about a month. All things considered I think I have a good excuse for being a little slow!

First off, at the prodding of Won Chun quat4 is now simply quat, because a quaternion implies 4 components and that makes it the same number of characters as the rest of the types.

Secondly, I'm curious what glMatrix users think about this idea: I'm considering renaming all of the types in 2.0 to have an upper-case first letter, as in mat4.multiply would become Mat4.multiply. The reasoning is that the old an new libraries will have functions that look extremely similar but have the arguments in a different order. Thus if you're trying to update a code base, you could easily miss several calls and end up with frustrating-to-track bugs. If the names are subtly different, however, it's easy to search for all instances that haven't been upgraded yet, and the browser can give you much earlier warnings about broken functions.

I don't know how common that scenario will be, but I don't find the change objectionable (I would normally case a class like that anyway, so it feels natural for "namespaces" like these). Does anyone else feel strongly about it?

[525c1e21-bd67-4735-ac99-b4b0e5262290]

Please please please try to be as consistant as possible. After all, besides performance, that's why we're choosing to come together with this project, is it not?

I'm slightly reluctant to change the name just for the sake of changing it.

vs.

I'm considering renaming all of the types in 2.0 to have an upper-case first letter, as in mat4.multiply would become Mat4.multiply.

---

If you're not entirely sure...

I don't know how common that scenario will be, but I don't find the change objectionable

...and the change can be emulated by the end user using something as simple as an alias, do not make the change. Save it for the footnotes, hints and tips or gotchas section of the docs.

---

That said I do appreciate and agree with the vast majority of changes being made.

Let us focus on catering for the 95% use case over the myriad of hypothetical what-ifs and but-thens.

---

CoffeeScript

As for CoffeeScript, see http://github.com/feisty/math

It's kind of old and incomplete now and there's things I'd change about the way it's structured and how things are named but there's the gist of what porting gl-matrix to CoffeeScript would involve.

IMO it's not worth straying from pure JS for something as fundamental as a 3d math library.

But it would certainly be nice to have compatible implementations, in languages other than JavaScript such as CoffeeScript (there's a TypeScript implementation here https://github.com/vexator/TSM).

---

A few things that bother me:

building / dependency management

lib contains ruby helpers to build the project. This is really confusing. If anything lib should contain the code which is currently in src. All this ruby can easily be crammed together into the Rakefile. It's just a build script.

On that note, IMHO sprockets really sucks. Consider using a dependency pattern and build system based on something like CommonJS modules i.e. require()

package

The project was never packaged and published to to repos like npm and bower. It's a trivial task with massive benefits. Please fix this. It's one of the only reasons I don't always use or recommend the library. I'm always forking and hacking the living daylight out of gl-matrix to beat it into shape for npm :(.

I should be able to do something like this:

$ sudo npm -g install gl-matrix
$ node
> var glm = require('gl-matrix');
{
  mat4: {...}
  vec3: {...}
  ...
}

---

Nevertheless kudos for the fantastic work and the new direction of the project. Keep it up!

[ioquatix]

IMHO type names should always be camel case.

[525c1e21-bd67-4735-ac99-b4b0e5262290]

camelCase or CapitalisedCamelCase?

Technically they're not really "types" (prototypes) though since they're not used in conjunction with new.

mat4.create() vs new Mat4

[ioquatix]

CamelCase as normally defined with the first letter being upper case.

[525c1e21-bd67-4735-ac99-b4b0e5262290]

That's generally called Pascal Case or CamelCaps http://en.wikipedia.org/wiki/CamelCase

Nevertheless the "modules" (mat{3,4}, vec{2,3,4}, etc.) of gl-matrix aren't really the traditional "types" (prototypes) you find in JavaScript such as Dates or RegExps. In the JavaScript universe, PascalCase is reserved for functions used in conjunction with new such as new Date() or new RegExp().

You will find it very difficult to locate an exemption to this policy in any standard JavaScript APIs. I personally do not know of any.

I'm not sure the container objects / namespaces / whatever you want to call them of gl-matrix which are not used as constructors in conjunction with new should be capitalised.

It might "feel" better or feel like you are promoting the "types" in gl-matrix to some elevated status but it will be flying in the face of 17 years of standard JavaScript naming conventions.

[ioquatix]

@pyrotechnick The first line of that article is "CamelCase (or camel case), also known as medial capitals or Pascal case".

I understand where you are coming from but can you point to any official documentation in the JS standards or specifications that support your opinions? In particular, I've never seen this statement anywhere before: "In the JavaScript universe, PascalCase is reserved for functions used in conjunction with new such as new Date() or new RegExp()"

In general, I've seen most APIs using CamelCase for namespaces, classes and I would find it hard NOT to call vec3 a type, I mean, clearly we are classifying some specific data structure with a name, in my mind this would be typically referred to as a data type, or simply a type.

[525c1e21-bd67-4735-ac99-b4b0e5262290]

Define official documentation...

http://javascript.crockford.com/code.html

Most variables and functions should start with a lower case letter. Constructor functions which must be used with the new prefix should start with a capital letter. JavaScript issues neither a compile-time warning nor a run-time warning if a required new is omitted. Bad things can happen if new is not used, so the capitalization convention is the only defense we have.

http://google-styleguide.googlecode.com/svn/trunk/javascriptguide.xml#Naming

In general, use functionNamesLikeThis, variableNamesLikeThis, ClassNamesLikeThis, EnumNamesLikeThis, methodNamesLikeThis, and SYMBOLIC_CONSTANTS_LIKE_THIS.

http://en.wikipedia.org/wiki/Naming_convention_(programming)#JavaScript

The built-in JavaScript libraries use the same naming conventions as Java. Classes use upper camel case (RegExp, TypeError, XMLHttpRequest, DOMObject) and methods use lower camel case (getElementById, getElementsByTagNameNS, createCDATASection). In order to be consistent most JavaScript developers follow these conventions.[citation needed] See also: Douglas Crockford's conventions

https://github.com/styleguide/javascript

• Always use camelCase, never underscores.
• Follow @jashkenas's style. See the documentation for good examples.

Meh. Have fun struggling to justify your naming convention to every other JavaScripter on the planet.

http://bikeshed.com/

[ioquatix]

@pyrotechnick Great, thanks for the update, I'm sure Toji will find it helpful.

Is "mat4" a method or a class?

[525c1e21-bd67-4735-ac99-b4b0e5262290]

It's neither. There are neither classes nor methods in JavaScript; at least until v6 lands.

"mat4" and friends are merely properties of the root object of the gl-matrix library.

I'm just going to go ahead and agree to disagree on this one.

Happy coding :)

[ioquatix]

Okay, let me ask that question from a different perspective, in C, would mat4 be a function or a type?

[toji]

Well, I sparked more of a discussion with the casing question that I anticipated! After reading the comments on both sides of the fence, I think I'm going to stick with lowercase mat4 and friends vs. uppercase Mat4. This is primarily because, as @pyrotechnick pointed out, consistency is important and making changes to assist an edge case is rarely a good plan.

Also, @pyrotechnick: I really appreciate your feedback on other parts of the project. I feel terrible that I've never gotten around to making glMatrix into an npm package. You can be assured that it will be fixed for v2 (and probably for v1 in the meantime. I'm making myself a reminder now...)

As for the current build system: To be honest I hardly know Ruby, that's all been Colin's work thus far and in my opinion it's been working well. I'm not keen to muck with something that's not broken. I have been considering making a small Google App Engine service that builds custom glMatrix libs, though (Don't use quats? Leave 'em out!) and that would require a Python variant of the build script, so maybe down the road. I'll cross that bridge when I get there though.

Finally, I've never cared much for CoffeeScript, sorry. I'm not opposed to a port but I won't personally be doing it.

One last item to bring up for more general discussion: There were a couple of requests at the beginning of this thread for a way to transform arrays of vec3s in a single function call. I actually stumbled on the need to do something similar today so I decided to mock it up and get some feedback.

/**
 * Transforms an array of vec3s in-place with a mat4.
 * 4th vector component is implicitly '1'
 *
 * @param {Array} a the array of vectors to transform
 * @param {mat4} m matrix to transform with
 * @param {Number} stride
 * @param {Number} offset
 * @param {Number} count Number of vec3s to transform
 * @returns {Array} a
 */
vec3Array.transformMat4 = function(a, m, stride, offset, count) {
    var i, l, x, y, z;
    var m0 = m[0], m1 = m[1], m2 = m[2],
        m4 = m[4], m5 = m[5], m6 = m[6],
        m8 = m[8], m9 = m[9], m10 = m[10],
        m12 = m[12], m13 = m[13], m14 = m[14];

    if(!stride) stride = 3;
    if(!offset) offset = 0;

    if(count) {
        l = Math.min(count * stride, a.length);
    } else {
        l = a.length;
    }

    for(i = offset; i < l; i += stride) {
        x = a[i]; y = a[i+1]; z = a[i+2];
        a[i] = m0 * x + m4 * y + m8 * z + m12;
        a[i+1] = m1 * x + m5 * y + m9 * z + m13;
        a[i+2] = m2 * x + m6 * y + m10 * z + m14;
    }

    return a;
};

The big question I have is if this setup provides the flexibility needed for the function to be useful? Also, does the order of the parameters feel sensible?

It also got me thinking that it would be valuable to have a more generalized version of this to easily do things like updating particle systems and whatnot, so I'm also considering having something like this:

/**
 * Perform some operation over an array of vec3s.
 *
 * @param {Array} a the array of vectors to iterate over
 * @param {Function} op Function to call for each vector in the array
 * @param {Number} stride
 * @param {Number} offset
 * @param {Number} count Number of vec3s to iterate over
 * @returns {Array} a
 */
vec3Array.forEach = function(a, op, stride, offset, count) {
    var i, j, l, x, y, z;
    var vec = Float32Array(3);

    if(!stride) stride = 3;
    if(!offset) offset = 0;

    if(count) {
        l = Math.min(count * stride, a.length);
    } else {
        l = a.length;
    }

    for(j = 0, i = offset; i < l; ++j, i += stride) {
        vec[0] = a[i]; vec[1] = a[i+1]; vec[2] = a[i+2];
        op(vec, j);
        a[i] = vec[0]; a[i+1] = vec[1]; a[i+2] = vec[2];
    }

    return a;
};

(And obviously vec2 and vec4 variants of the same) It's worth noting that I wanted to use Float32Array.subarray for the inner loop rather than copying values back and forth, but apparently that's painfully expensive

[525c1e21-bd67-4735-ac99-b4b0e5262290]

Thankyou for your sound decisions @toji

You can have your thread back now :)

[sinisterchipmunk]

Sorry I've been out of the loop. Been rather busy, as most everyone seems to be these days.

@pyrotechnick

Thanks for the input. I agree with a lot of your points but feel I have to defend myself on the build system...

For the build system, I used Ruby because I was comfortable with it, and because I know how to make it interface with just about any other tool or build system we decide to evolve towards. I'm not against switching to a different system entirely, but I like the clarity and conciseness offered by Ruby, and nobody objected at the time. :)

For calling lib what it is, that decision was made as a matter of convention. I've never not seen a project put its supporting Ruby files in lib, but I don't deny that it's done, and I don't have anything against renaming the directory if Brandon agrees there is need to do so. We could call it build or something.

I'm flatly against the idea of moving all the Ruby code into the Rakefile, though. Code is code, build script or not, and readability and maintainability of code are of paramount importance IMHO. The idea of cramming it all into one file makes me literally cringe (and I'm not exaggerating! I take a lot of pride in writing clean code). Also, cramming it all into one file seems a lot like cramming all the JS into one file, something we've specifically taken steps against in v2.0.

On that note, IMHO sprockets really sucks.

I'd love to hear some specifics on this point. When I first started working with Sprockets, I had the same feeling, but have come to understand that most of my problems were due to a lack of understanding of the library itself. It does have some real strengths and major benefits, though they aren't really utilized in this project, and I've come to respect Sprockets a great deal.

@toji

Although I like the gist of what's happening in vec3Array.transformMat4, I'm still uncomfortable. It feels like a slippery slope into a ton of extra code (e.g. vec2Array.transformMat3, vec3Array.transformQuat and other variants) that will need to be maintained. It is starting to feel like bloat to me. OTOH, I really like the forEach idea, because it's both readable and versatile, even though it brings much the same code maintenance to the table. In the latter case, however, the overhead of the extra function call makes me a little nervous, but that might be unfounded as I haven't seen any recent benchmarks testing the cost of a function call. The other thing that makes me nervous is the callback itself. They're probably one of the most powerful features of JavaScript, but they're dangerous in a performance-driven environment like a WebGL app, because most people don't bother to cache the reference to the function. They just create functions in every frame. Still, that's more their problem than ours, and the same can be said for caching vectors and matrices. I'm generally in favor of one or both of these methods, as long as the proper usage is well documented in order to help inform people who are new to writing performance apps.

I do have a qualm with the argument order, but it comes down to preference. I have always had trouble with code like this:

setTimeout(function() {
  // do some stuff
}, 1000);

because to my eye, it's easy to miss the second argument and just feels "wrong". (And don't get me started on functions that accept multiple callbacks!) Granted, if you're caching the function reference it's less of an issue, but it still bugs me. Guess that's just how I am.

So, if you move op to the end of the arguments list, stride, count and offset become mandatory arguments. I'm not super enthusiastic about that idea either, but making them mandatory would carry an ever-so-slight performance boost, so there's a silver lining I suppose.

In the end I'll be on board with @toji decides, but I wanted to make sure these things are discussed.

[ioquatix]

@sinisterchipmunk

Actually I did originally suggest putting together some build scripts. I've got an existing track record of Ruby based build scripts for JS projects so I've had some real experience with how to do this process.

Putting the Ruby code in lib for what amounts to a non-ruby project isn't really a good idea. I'd recommend making a master rakefile and having a directory called tasks where you put individual tasks. You can see what I'd recommend on my dream-framework which does essentially that.

Finally, while it is common, I don't recommend that people check out code directly into their application directory. Instead you'd be better of a with a build system that can copy the files somewhere else based on the configuration file. That config file can specify what parts of the library to use and whether to minify, etc.

I don't think its advisable to be checking a minified copy of the library directly into the source code tree, and having some online system to do it just seems tedious in a well engineered project.

Secondly, my original suggestion regarding array based operations is primarily centred on common operations where you can improve performance by avoiding temporary object construction.

If you have done any serious 3D programming, transforming Vec3 vertices by a Mat44 is one of the most common operations and should be optimised if possible, second only to Mat44 * Mat44 multiplications. Therefore, including an optimisation for this specific case would be very useful in general. Other cases like Vec2 x Mat44 are less useful in general.

I hate to say it but this thread reminds me of the phrase "Too many cooks spoil the broth". I think the discussion is probably pretty useful but I can't help but think this whole thing has started to become really over designed - for what its worth I'm probably just going to roll my own opinionated math library sometime soon :D

[sinisterchipmunk]

I'd recommend making a master rakefile and having a directory called tasks where you put individual tasks

This is the norm for Rakefiles with lots of tasks; it's done for Rails and I do it in Jax. I have only ever added actual Rake tasks to tasks and not supporting code for them, but in hindsight it does make sense, especially for a non-Ruby library where code reuse is less of an issue. As stated previously, I'm not against the idea of moving the Ruby files out of lib, and tasks sounds like a good place for it.

I don't think its advisable to be checking a minified copy of the library directly into the source code tree

In general I would agree. Minification used to be a painful black art, but it is getting easier to do; Rails does it automatically and anyone in the dark can check the Rakefile to see how to do it in 1 line with Ruby. I think this project is gradually evolving away from the need for a committed minified copy. Having said that, I don't know that we're actually to the point yet where everyone can minify on their own. Sometimes people need a hosted minified version for one reason or another. Maybe it could be moved to GitHub Pages and an extra step in the release task, but I feel that decision would have to be @toji's as he pretty much is in charge of releases.

If you have done any serious 3D programming, transforming Vec3 vertices by a Mat44 is one of the most common operations and should be optimised if possible

Agree 150%. I only wanted to point out that implementing it could segue into tons of additional rarely-used functions, and that by the nature of the math it would really imply maintaining a minimum of two variants, one for vec3 and another for vec4.

Don't get me wrong -- it could be nice to have those rarely-used functions ready for when they actually are needed, so I wasn't voicing an opinion for or against, only bringing discussion about it to the table.

I hate to say it but this thread reminds me of the phrase "Too many cooks spoil the broth".

I think the discussion is healthy. It's always good to hear arguments other than our own. When it gets bad is when we stop listening to each other, and I don't think this thread or this project has degenerated to anywhere near that point. I think we're still in the "healthy discussion" stage.

I can't help but think this whole thing has started to become really over designed

Well, it's a feature discussion, not a specification, meaning that not everything discussed here is going to become part of the library. If you ignore the above conversations and look solely at what's actually being committed, personally I have a good feeling about the architecture and direction of the project. That we're spending more time on relatively small things like directory structure indicates to me that we don't have any bigger issues to worry about, which I find very comforting.

[ioquatix]

@sinisterchipmunk Super :D

[525c1e21-bd67-4735-ac99-b4b0e5262290]

Question: do we even need a builder?

100% of the time I use gl-matrix I do so through a package manager like npm, component or bower.

Once you've got the library into such a system you can leverage other projects which build, compile, minify and otherwise contort the code to your own particular likings.

If people want a combined and/or minified build:

• write a tiny script and keep it in a gist
• put it on the GitHub page for the project and link to it
• put it in the "downloads" section and link to it

Everywhere the users who want the easy way out will possibly look to find it.

I believe we'll all find it incredibly hard to agree on a build system because it's subjective to taste and application.

Let's focus on what we all actually desire from this project:

• consistency
• performance

How about populating a wishlist for v2 with concrete, immediately obvious features and aspects we believe the vast majority of us want from the project?

And then let's get to crossing them all off as quickly as humanly possible.

[sinisterchipmunk]

100% of the time I use gl-matrix I do so through a package manager like npm, component or bower

Very good point, and this approach may lend itself to Brandon's earlier statement about picking and choosing components (e.g. quats).

I was already considering writing a Ruby gem to wrap around gl-matrix and drop it into Rails' asset management. (Maybe it's out there already; I haven't checked.) The gem would automate the build process in the Rails world too, so that's one less place we would need a builder.

How about populating a wishlist for v2 with concrete, immediately obvious features

Another smart idea. This discussion is great and all but it's hard to track what we've actually decided. The only thing I wonder about is whether we need a centralized list, or if this issue tracker itself with a 2.0 milestone attached to each decision and maybe a reference to this thread is good enough.