Technology Overhaul

[Usnul]

I have been following three.js for many years now, as well as using it in my own projects. I have watched it develop into the amazing library and community surrounding it that it is today. There are many amazing things in three.js, but overall, as far as state of the art for real-time rendering engines goes - three.js has remained relatively stagnant for about 4 years now. Therefore, i would like to list some of the features i'd like to see in three.js in the future, and the reasons I believe them to be necessary for continued success of this library.

Deferred rendering

Forward rendering doesn't fit well with most advanced use cases, and when it does - it lacks performance. I want to see a Deferred, Tiled renderer as the main renderer in three.js, not something relegated to /examples folder.

Motivation

Any kind of post processing you want to have today requires you to build part of the pipeline, depth buffer, normals and colour also. This results in duplicated effort for the hardware in most cases resulting in sub-optimal performance. If you wish to have any kind of sub-system which relies on a G-buffer, such as G-buffer based decal system which writes into colour+normal buffers - you require what three.js doesn't have.

Better Shadows.

More specifically, i want to see a better shadow-mapping implementation than PCF. Such as Variance shadowmaps. Second, i wish to see a cascading shadowmaps. They were once at the core of thee.js, but fell victim to bitrot and lack of support.

Motivation

Currently shadows require a fair amount of artistry just to make them look okay. If we had a better shadowmap algorithm - a lot of existing problems with acne and filtering would go away. If we had a cascading shadowmap implementation - it would remove the need for artistry currently required to set up shadow camera for each shadow-casting light in the scene.

Spatial Index

Spatial index is necessary when dealing with large scenes, such large scenes are very common in games for example.

Motivation

If you want to do a raycast into the scene - currently you are stuck with a linear search, which is dominated by number of objects and polygons. A spatial index would enable a lot of internal optimizations, such as faster occlusion culling and sorting.

Occlusion culling

Good occlusion culling is required for good performance. Point above would help here. There are a lot of techniques that can be utilized further here.

Optimizations to animation engine

Currently animation engine chokes on some 500 bones being animated simultaneously, resulting in a very high CPU usage.

Motivation

It is not uncommon to see 3-5 characters at the same time with 500+ bones each in modern games, with current CPU demand such fidelity is not achievable, instead you have to compromise to about 15 bones per character in order to achieve decent performance.

Compressed Textures

Compressed textures as a first-class citizen, along with tools for on-line compression

Motivation

Compressed textures offer a great amount of extra detail requiring only a little space, for applications with large textures and/or large number of textures, this draws a line between interactive frame-rate and a slide-show, this point becomes more relevant for lower-end GPUs, as they tend to have less RAM, being able to draw 2024 compressed textures instead of 512 uncompressed ones is a extremely important, as they take up potentially the same amount of GPU RAM. Compressed textures take less time to load and put less stress on browser, since decompression is not done by default (unlike PNG).

Competent Particle Engine (nice to have)

Particles are the magic stuff of the real-time visualization. As such, i believe it is necessary for a comprehensive real-time visualization to have a good particle engine. Simulation aspect can be kept rudimentary, have a way to plug in simulation logic, but handle things like sorting particles in view-space according to depth, and offer solution for tracking lifecycle of a particle, such as spawning/dying. Things like self-shadowing could be amazing to have also.

[RemusMar]

Good points (and motivations). :+1:

Here is another one: many game developers are interested in a physics engine for Three.js The current implementations are primitive and waste a bunch of resources.

[Mugen87]

three.js has remained relatively stagnant for about 4 years now

I don't agree with that statement. In that time period, a lot of features were added which are valuable for many projects:

• PBR materials (MeshStandardMaterial/MeshPhysicalMaterial)
• WebVR related entities
• Improved quality/features of importer/exporters like GLTFLoader/GLTFExporter, FBXLoader or ColladaLoader
• Modularization of WebGLRenderer
• And many more. Just read the release notes.

All these stuff required a lot of development effort. Please do not misunderstand me, I think you've made valid suggestions in your post. But saying three.js has remained relatively stagnant for about 4 years is not really fair.

[fernandojsg]

I agree with @Mugen87 apart from working on new features as he exposed, just keeping up do date with compatibility between different browsers and platform and fixing bugs is a huge work for a project that is basically growing up with the open source community without any big company behind. So it's something really awesome what @mrdoob and the community have been doing so far.

I also agree with the idea from @Usnul that we need to keep adding new features to the engine but actually modularizing the renderer is something needed to include some of these changes. As for example with the deferred rendering, that's something I see as optional, as not everyone will take that as a the default, if I need to choose I would prefer something like a tile-clustered forward rendering, so you don't want to mess with the core renderer for each option.

In any case I would suggest to create single issues for each feature so we could keep the discussion there more focused on the specific feature. And as it seems that no one is working right now on these features feel free to pick one you want to work and submit a PR it will be really valuable for the engine.

[donmccurdy]

+1 for separate issues on each feature. Perhaps keep spatial index and occlusion culling together. Thank you for the suggestions — I think this will be valuable to discuss.

[titansoftime]

I agree that Shadows leave a lot to be desired. Performance right now is not so great + acne. Self shadowing is hard to pull off without crazy artifacts. Cascading shadows would be incredibly awesome.

Yes, the animation system is a CPU mega hog. I had to resort to only animating skinned meshes within the camera frustum among other little hacks to get performance to a tolerable point.

One feature request I would like to see, is a simpler API for instanced geometry. One that does not require knowing glsl.

Regarding the particle engine. This was created by @squarefeet and has more than met my needs. This should be added to three library imo (kinda how like @jbouny's ocean was added). It rocks. https://github.com/squarefeet/ShaderParticleEngine

I use the DDSLoader for compressed textures, works great. Curious as to what you mean.

That being said, I absolutely LOVE three.js, and though it's not meant to be a game library, it sure works great as one. I have a fully playable browser based mmo running on my phone (droid, because skeletal animations are broken for apple devices) at 40fps with decent graphics. Not bad.

[Usnul]

Wow, a lot of feedback. Thanks guys. I did not mean to say "three.js sucks and nothing new was added", I'm sorry if I came across like that. I too use three.js with great success, and I believe it is the best non-commercial web-based renderer out there.

@titansoftime Regarding DDS. It's true that you can load DDS textures, but you can't do so with say JSON loader which comes bundled with core, at least not without hacks. Lets say I load a png texture, but I want to re-compress it into something that GPU would be able to work with - right now your options are limited to off-line conversion only. And tools like crunch are great, but you have to know about them, and their usability is quite restricted. I would like to have an option to convert a few valid texture formats to DDS seamlessly, this doesn't have to offer best compression like crunch, and doesn't have to be done instantly in the Main Thread.

@RemusMar Regarding physics. I considered this. My own experience with physics engines left me thinking that it's a lot of work. First, you have to pick a physics engine to integrate, second - I still ended up managing bodies and renderables separately, so it felt like a decoupled issue to me.

PS

do you want me to create separate issues?

[RemusMar]

Regarding physics. I considered this. My own experience with physics engines left me thinking that it's a lot of work. First, you have to pick a physics engine to integrate, second - I still ended up managing bodies and renderables separately, so it felt like a decoupled issue to me.

The best approach is embedded physics engine. Here is a sample: 3D engine + physics engine = a single JS file of 232 KB !!! http://necromanthus.com/Games/HTML5/cc/test.html Quite impressive for JS and WebGL. cheers

[Usnul]

@RemusMar Problem of "embedded" physics engine is the limitations of physics engine. This is a project of real-time 3d graphics, with a bit of a focus on games. Physics engines such as bullet, on the other hand, focus on physics and simulation. Having both domains in one project would reduce focus or require an infusion of fresh blood with that additional background. There are quite a few 3d physics engines available in js nowdays - they have different limitations and performance profiles. Let me give you a few examples:

• constraint joints
• support of mesh-based shapes (not just basic geometry like cube/sphere/cylinder)
• support of height maps
• inverse kinematics
• cloth
• soft body dynamics
• collision bodies without response (detectors)

[RemusMar]

There are quite a few 3d physics engines available in js nowdays

I know them all and as I said before, they are not properly implemented (not good for WebGL and browsers). Talking about Three.js: "The aim of the project is to create an easy to use, lightweight, 3D library." I subscribe to that.

[Usnul]

@RemusMar

What would be your proposal?

• create a new physics engine as part of three.js?
• integrate an existing physics engine?
  • as is, synchronize on release cycles?
  • fork and adapt?

What do you mean by "not properly implemented (not good for WebGL and browsers)"?

What do you think of bullet?

My best experience so far was using bullet (ammo.js) with a custom Worker binding and memory management.

[RemusMar]

What would be your proposal? integrate an existing physics engine?

For Three.js that wouild be probably the best solution.

What do you mean by "not properly implemented (not good for WebGL and browsers)"?

Huge file size and significant performances drop. Compared to them, please check out the above link (232KB and "all in one"). That's the best solution so far talking about file size. Talking about performances, the path to go on would be something similar with Nvidia PhysX and GPU acceleration.

What do you think of bullet? My best experience so far was using bullet (ammo.js) with a custom Worker binding and memory management.

An acceptable solution (but far away from optimal)

[Usnul]

@RemusMar I checked out the link you posted, to the 232KB "all in one" engine. It's quite impressive for what it is, I especially liked the foliage, it looks really awesome. But at the same time it demonstrates a very limited set of features. Just as an example, my own engine sports some 40-odd systems; to name some features: sound, animation, path finding, compressing, spatial index (BVH and r-tree), visibility sets, particle engine, physics, heads-up-displays, tiled terrain, foliage, montecarlo AI core, concurrency engine, serialization/deserialization, asset management

problem is - to have a competent game engine, it requires bringing a lot of knowledge from fairly disparate domains together. In game engine literature "ECS" is often given as an example of game engine architecture - i feel it is misleading, since ECS only covers one aspect of what makes a game engine - the simulation.

In modern games usually you end up with a lot of so-called "middleware" components, each of which handles a specific task, and does so very well. Like sound(e.g. wwise), path finding (e.g. PathEngine), occlusion (e.g. Umbra), LOD (simplygon) etc. I'm not sure if this is just a trend, or an evolution of the industry. Game engines such as Unity or Unreal come with a ready bag of well integrated tools that cover a lot of concerns, but often they can't compete with highly specialized middleware providers, due to accumulated knowledge, patents, required effort etc.

[donmccurdy]

Just FYI — no problem with having a holistic initial conversation here if it's helpful to set direction, but I am not planning to give any specific feedback in this thread. I think it would get lost and be hard to follow in the future if we get too much detail here, "technology overhaul" is very broad, so I am planning to wait until issue-specific threads are started before commenting further.

[Mugen87]

Just FYI — no problem with having a holistic initial conversation here if it's helpful to set direction, but I am not planning to give any specific feedback in this thread.

Same here. And when issues are created, people should bear in mind that it is not the goal of the project to create a game engine.

[titansoftime]

Same here. And when issues are created, people should bear in mind that it is not the goal of the project to create a game engine.

So I chose the wrong library then?

[donmccurdy]

Same here. And when issues are created, people should bear in mind that it is not the goal of the project to create a game engine.

So I chose the wrong library then?

When three.js claims to be a “3d library,” this means (IMO) that we want to leave users the flexibility to use the library for games and also for data visualization, generative art, simple model viewers, and so on. If you’re using three.js as a game engine, you will want to make opinionated choices about which physics engine to use, which materials and effects to allow, how to manage state, how large the build can be, and more. We do not make those choices for you, but consequently we also can’t provide an all-in-one game engine. Unity or BabylonJS are good products for that purpose.

That doesn’t mean we won’t spend time on game-related things; we do. But some features require opinionated choices about how apps are built and should be done on top of, not within, three.js. I hope it's clear that I'm not trying to devalue the suggestions in this thread — many of them are totally reasonable requests given three.js's scope. But it is complicated, and I don't think we can reach a decision in a single thread.

[titansoftime]

@donmccurdy Thank you for that. As a game dev who chose three.js (for years now), I'd hate to be disenfranchised.

I personally do not need a bundled physics engine or anything like that, just performant skinning and decent shadows =]

I do really appreciate the hard work all the contributes have put into this project. It is super cool.

[takahirox]

FYI, THREEx Game Extensions for Three.js http://www.threejsgames.com/extensions/

[titansoftime]

@takahirox If that grass extension works as advertised, I am going to track you down and marry you (no not really, relax) =]

My world could use it:

[RemusMar]

people should bear in mind that it is not the goal of the project to create a game engine.

To Michael ( @Mugen87 ) and Don ( @donmccurdy ): Game engine and 3D engine are different things. Three.js (and Babylon.js) are 3D engines. cheers

p.s.

That doesn’t mean we won’t spend time on game-related things;

That's the main idea in this topic.

[Mugen87]

But it is complicated, and I don't think we can reach a decision in a single thread.

Because of this, I vote for closing the issue. This discussion leads to nothing...

[RemusMar]

This discussion leads to nothing...

Maybe. But the goal of this discussion is not a decision. There are 2 big problems with the open source projects with many collaborators and hundreds of "contributors": 1) the lack of vision 2) good for anything = great for nothing

Some people here want a more specialized 3D library. Other people (Mugen87, looeee with his :+1: ) want a general purpose 3D library. It's up to Ricardo to find the best compromise.

just my 2 cents

[looeee]

@RemusMar you're reading a lot into a 👍... 😛 I was agreeing with @Mugen87 that the conversation is going nowhere and this issue should be closed, that is all.

There were some interesting points raised by @Usnul in his initial comment. Better shadows and an improved animation engine in particular I'd love to see.

But these issues deserve their own threads. I don't think we'll gain anything by continuing this one.

[RemusMar]

RemusMar you're reading a lot into a 👍..

I see a lot of 👍 between the collaborators. :laughing:

Better shadows and an improved animation engine in particular I'd love to see.

:+1:

Again, this topic is about "general purpose" vs "more specialized" 3D library. At least from my point of view.

[Usnul]

I finally got to creating the individual issues. I hope this discussion will continue there :)

[titansoftime]

Wow, you aren't messing around =]

[Mugen87]

Closing in favor of the newly created issue.

[donmccurdy]

Thanks for starting the individual issues! 🙂