Add motion blur pass

[marcofugaro]

Recently I've hacked the motion blur pass from gkjohnson/threejs-sandbox and this PR to work with the postprocessing library.

It would be great if this was supported as a first class pass and included in postprocessing, with @gkjohnson approval of course.

I could make a PR but I'd need the help of @vanruesc to adapt the code to the repo standard and maybe tweak the render workflow.

[gkjohnson]

Hey @marcofugaro! I've toyed with the idea of maintaining a package of some of the effects in my sandbox repo but I'm finding I don't have the time with everything else. If adding it to this package enables some nicer looking WebGL projects then that sounds great to me! A small attribution at the top of the file saying where the effect was adapted from would be nice but not necessarily required.

I'm not sure which version of the effect you've used but I recently updated the effect to include a couple different "smear" sampling approaches which can greatly improve the quality and can let you get by with far fewer passes. Specifically using blue noise sampling seems to look the best to me but perhaps the random sampling used elsewhere in this project is good enough.

A couple other things that might improve the effect if there's an easy way to add them here:

• Transparent objects could be drawn last after motion blur has been been performed otherwise they will just blur the objects behind them depending on their opacity. Of course this means that transparent objects would not get motion blur applied. This might be something the end user has to do, though.

• Blur samples could be optionally discarded (or weighted) based on how far in front of the estimated previous position the depth buffer is otherwise objects in the foreground can "bleed" into the motion blur.

The improvement from the above two changes might be minimal but I've had them in mind. If it turns out there's interest in adding this I'm happy to help out with whatever questions you have. I'd like to learn a bit more about extending this project, as well.

[marcofugaro]

Specifically using blue noise sampling seems to look the best to me but perhaps the random sampling used elsewhere in this project is good enough.

I tried the new version with smearing, and I found out that the random smearing is kinda noticeable with the naked eye, but maybe is good for some other use cases. I resorted to the bluenoise or just plain high samples.

Blur samples could be optionally discarded (or weighted) based on how far in front of the estimated previous position the depth buffer is otherwise objects in the foreground can "bleed" into the motion blur.

Nice!

If it turns out there's interest in adding this I'm happy to help out with whatever questions you have. I'd like to learn a bit more about extending this project, as well.

By all means, you can make the PR as well if you want, I just want to be able to use the MotionBlur pass with this easy-to-use library.

It just needs some direction from @vanruesc since he is the main maintainer of the project and he will be maintaining the pass. Hope he is interested in including this pass as well.

For example, I wanted to ask, what is the corresponding method of doing new Pass.FullScreenQuad() in postprocessing?

https://github.com/gkjohnson/threejs-sandbox/blob/2590e4978f0cac023b65e01086ccef0b8d8bb2a4/motionBlurPass/src/MotionBlurPass.js#L129

[vanruesc]

My current plan is to add a central buffer management system to postprocessing in the near future if time permits. I'd also like to convert the project to TypeScript. Passes and effects will receive a new input/output system for render targets and uniforms and I want to remove all deprecated methods, classes and fields before finally releasing a new major version.

These are big changes, so I think it would be better to wait until that's done before adding more complex effects. The goal of these changes is to make it easier for people to contribute.

Although I personally don't like motion blur, I'm not opposed to adding it to the library. I saw that the implementation uses some of three's built-in shader chunks which could complicate things later on. We might want to wait for NodeMaterials before adding this effect.

what is the corresponding method of doing new Pass.FullScreenQuad() in postprocessing?

You'd just set a fullscreen material with Pass.setFullscreenMaterial(material) and the pass would then use the shared internal fullscreen triangle. However, the composite shader should actually be implemented as an effect since its a fullscreen effect.

[marcofugaro]

These are big changes, so I think it would be better to wait until that's done before adding more complex effects.

Alright, thanks for maintaining this repo.

I saw that the implementation uses some of three's built-in shader chunks which could complicate things later on. We might want to wait for NodeMaterials before adding this effect.

We could just include the chunks code in the shader in the meantime, and add a comment to which chunk the code is copied from, for future improvements.

[gkjohnson]

@marcofugaro

By all means, you can make the PR as well if you want, I just want to be able to use the MotionBlur pass with this easy-to-use library.

I don't think I'll have the bandwidth to do it myself but I'm happy to follow along when the PR is made and answer any questions!

I saw that the implementation uses some of three's built-in shader chunks which could complicate things later on.

I believe all of the used shader chunks are just required for skinned meshes to compute the pixel movement from the previous frame. A first pass could not include support for skinned animation velocity.

[unphased]

I'm interested in helping with the process of integrating motion blur to this library! Motion blur has a special place in my heart as I cut my teeth on advanced shader effects with it when I added rigid motion blur to my homegrown box2d engine over 10 years ago. This is probably my proudest achievement from my teenage years.

Fragment_shader_dynamic_blur.pdf

While motion blur gets a bad rap because it's often abused in games, functionally detracting from clarity and adding control latency, especially when implemented poorly (even as a motion blur "enthusiast" I'll eventually turn it off in half the games I play, would be good for them to let me enable it for objects but tone it down for the camera!), I believe that tastefully/subtly applying motion blur goes a long way toward making a 3d scene look realistic (short of 240hz+ display and corresponding framerate). It's impossible to make up that ground in terms of latency on a 60hz display, but in terms of perceived fluidity, motion blur can add a lot. Latency and temporal resolution can and perhaps should be decoupled, just as it is widely accepted now that animation and rendering rates should be decoupled.

@gkjohnson I hadn't seen your demo until now but I am blown away, as that is a very high quality implementation of motion blur in 3D and in my opinion it looks every bit as good as the best AAA game implementations I've seen out there. We probably pay a bit of a tax implementing it on top of three.js, as optimizing performance for this sort of advanced technique is best done with an engine built around it, or at the very least a deferred architecture! But at least it runs like a dream on my RTX3080! Haha. I'm just so impressed by how far three.js has come.

The thing that I realized 12 years ago was that motion blur disproportionately benefits the clarity of perception of small objects that are moving fast. I don't think I particularly chose the domino tower dynamics for highlighting this, but it does it well. It's even more apparent when you use the box2d mousejoint (click on an object and drag it around) and fling around the little pieces. With 50 samples of motion blur it creates an effective framerate of 60 * 50 = 3000 fps of temporal resolution, and even higher on modern high refreshrate displays. This is capable of showing tiny nuances in motion in the same way that they are perceived in real life. Notice how the box is seen to be rotating in the wrong direction without the blur. I also find the perfectly reproduced curves of motion very pleasing to the eye, and this is something that traditional rendering still largely lacks.

Actually now that I think of it I don't think I've seen a fully 3D implementation of my entirely 2D technique yet. It's a lot of tricky hurdles to scale just to get at perfect detail on fleeting motion of objects with particularly high angular velocities.

[Hoodgail]

Recently I've hacked the motion blur pass from gkjohnson/threejs-sandbox and this PR to work with the postprocessing library.

Could you please share the code? because i was using @gkjohnson 's motion blur shader, but i just moved to this project's library, i didnt know how i'd be able to get it to work with this lib

[kitaedesigns]

Hey all, is this going to get added anytime soon or is it dead?

[ElMehdiBouamama]

We need this 🆙

[vanruesc]

@kitaedesigns This hasn't been added yet because v6 doesn't provide the tools to implement it efficiently. With v7 mostly ready, we can now start working on this.

Closing in favor of #577 and #582.