oculus rift support, aka multiple cameras

[alon]

There is a single camera per window right now. I propose turning that into cameras, for the stereo side by side rendering use case that I'm interested in, but this also works for things like mini maps. What do you think?

note that for the OR (oculus rift) I also need to apply shaders to undo the pin-cushion effect, see: http://www.dgp.toronto.edu/~mccrae/projects/firebox/

I plan to try to fix this issue myself unless you were already planning to :)

[sebcrozet]

That sounds like an amazing feature. I'm not working on this at the moment so any contribution is very welcome!

However, the shader part might not be easy to implement since most shaders of kiss3d are hard coded. There is a ShaderManager but it is not able to handle more than the two pre-defined shaders, so there is no clean/easy way of switching between several shaders yet. I plan to improve that soon since this is blocking #1 anyway.

In the mean time, feel free to modify any hard-coded stuff by whatever you need to achieve your effect :)

[alon]

I've read a bit of the code since entering the issue, I'm not entirely sure what the right way to do it will be, but adding a Camera should be fine, since the idea is to have two rendering calls, like this:

set projection matrix set anti pin cushion pixel shader (left side parameters) render <- this renders only to left side, it sets pixel color to 0,0,0,0 for anything on the right side via the shader

set projection matrix 2 set anti pin cushion pixel shader (right side parameters) render <- right side only

I guess I'll add another hard coded shader for now since you are planning to fix it, and I'm thinking maybe having a special StereoSideBySideCamera instead of changing Window 1:n Camera, since there is projection / unprojection, and Camera is actually in charge of event handling too, so it might as well be in charge of the whole screen. But then the name makes no sense. Maybe I should introduce something in the middle. I'll start with the StereoSideBySideCamera.

On Mon, Sep 16, 2013 at 10:34 PM, Sébastien Crozet <notifications@github.com

wrote:

That sounds like an amazing feature. I'm not working on this at the moment so any contribution is very welcome!

However, the shader part might not be easy to implement since most shaders of kiss3d are hard coded. There is a ShaderManager but it is not able to handle more than the two pre-defined shaders, so there is no clean/easy way of switching between several shaders yet. I plan to improve that soon since this is blocking #1 https://github.com/sebcrozet/kiss3d/issues/1 anyway.

In the mean time, feel free to modify any hard-coded stuff by whatever you need to achieve your effect :)

— Reply to this email directly or view it on GitHubhttps://github.com/sebcrozet/kiss3d/issues/7#issuecomment-24537812 .

Alon Levy

[sebcrozet]

set projection matrix set anti pin cushion pixel shader (left side parameters) render <- this renders only to left side, it sets pixel color to 0,0,0,0 for anything on the right side via the shader

set projection matrix 2 set anti pin cushion pixel shader (right side parameters) render <- right side only

There are multiple limitations making this non-trivial to implement:

• a camera cannot tell the rendering engine to draw the scene, so this two-pass rendering has to be done by the Window.
• the Window stores the camera as a Trait-Object, meaning that there is no way for it to know that your camera needs two rendering passes. One solution could be to modify the camera interface, adding a fn num_passes() -> uint; method to ask for the number of passes needed to draw the scene, and change the fn update(&mut self, window: &glfw::Window); method to fn update(&mut self, curr_pass: uint, window: &glfw::Window);. This is obviously limited (and kind of ugly), but might work as a temporary solution.

Another solution could be to move all the rendering code (draw function) from the Window to the cameras. That way, a camera can do whatever it needs to render the scene, including multiple passes. I’m not sure this could be a durable solution, but this might be the simplest way to implement the StereoSideBySideCamera.

[...] changing Window 1:n Camera, since there is projection / unprojection,

If multiple camera can be active, the projection/unprojection methods of the Window wont make sense any more. I guess the user would have to choose one camera, and call its projection/unprojection methods directly.

and Camera is actually in charge of event handling too

This is clearly a design flaw. Events should be handled by a stand-alone controller able to modify any scene node. I just created an issue for that: #9.

Maybe I should introduce something in the middle. I'll start with the StereoSideBySideCamera.

Not sure I understand what you mean by "something in the middle" here.

To recapitulate, based on your propositions, here are the three possible solutions I have in mind:

1. create a StereoSideBySideCamera, modify the camera interface, and make the Window multipass rendering aware. This is not a durable solution.
2. make the Window handle n active cameras, and make one pass per camera. This will have some impact on projection/unprojection, event handling, and the user API to add cameras. Fell free to make the projection/unprojection fail until a viable solution is found.
3. give to the cameras the ability to render the scene, and modify the Camera interface accordingly. This might be the simplest solution. Somehow, this might be a durable solution too.

[alon]

See my patches here: https://gitorious.org/ocolus-juggling/kiss3d/commit/04be72447172c08d699bff26690e903a6380eb26

The last one is mainly the example and glsl, nothing much to review, the engine changes are the first (last here, since the top is the newest) few patches, starting with "window: add width/height setting constructor" and "window+camera: add variable passes, defaults to 1" and then using it in "add stereo side by side camera". I decided to copy first_person.rs to first_person_stereo.rs rather then making it a special case - I think it's better, since the stereo might gain more rendering options later on (side-by-side is just one of the existing standards, happens to be used by the oculus which beget it).

Please review!

Alon

Specifically: (dates are mixed up because of rebasing) commit 04be72447172c08d699bff26690e903a6380eb26 Author: Alon Levy alon@pobox.com Date: Fri Oct 4 19:50:15 2013 +0300

oculus_stereo as a clone of wave without dt update

shader is loaded from a file so can be changed without recompiling.

examples/stereo.rs | 47 ++++++++++++++++++++++------------ oculus_fragment_shader.glsl | 65 +++++++++++++++++++++++++++++++++++++++++++++++ oculus_vertex_shader.glsl | 9 +++++++ src/lib.rs | 1 + src/post_processing/oculus_stereo.rs | 158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 264 insertions(+), 16 deletions(-)

commit 4035c0da80bbe95b3612f4125a8e524e50cfb67e Author: Alon Levy alon@pobox.com Date: Thu Oct 10 01:33:29 2013 +0300

add stereo side by side camera

make sure it works with shaders too, see examples/stereo.rs

Makefile | 1 + examples/stereo.rs | 70 ++++++++++++++++++++++++++ src/camera/first_person_stereo.rs | 320 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/camera/mod.rs | 4 ++ src/window.rs | 1 + 5 files changed, 396 insertions(+)

commit 7f7b79567c37f9deb1e105dd1cd420027f9bc0df Author: Alon Levy alon@pobox.com Date: Thu Oct 10 01:30:57 2013 +0300

window+camera: add variable passes, defaults to 1

src/camera/camera.rs | 14 ++++++++++++-- src/window.rs | 24 +++++++++++++++--------- 2 files changed, 27 insertions(+), 11 deletions(-)

commit 784942d4760bce99c005115b7ef004e99b1b5892 Author: Alon Levy alon@pobox.com Date: Thu Oct 10 01:28:57 2013 +0300

window: add width/height setting constructor

src/window.rs | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)

[sebcrozet]

Great work, thanks! I merged your patches.

I agree with all your changes of the render loop and the camera trait, though I think this should be temporary, until a more evolved effect system is written.

I fixed a minor bug (to initialize the scissor when the window is resized) to make the other demos work correctly. I also modified the cameras upload_mat method to take a reference to the homogeneous matrix (to avoid an useless copy).

I do not think this is a good idea to load your post processing shaders from two files. Since the effect is part of the library, its shaders will have to be inside of the produced library binary directly: it would be annoying for the user to have to copy the shader files to its own project directory in order to make the effect work.

Also I noticed a visual artifact on the demo. When a cube is cut by one of the lens, a thin red line shows up on the other one. Here is a screenshot (the artifact is not visible on the miniature so you will have to click on it to see the original image); notice the thin vertical red line around the middle of the screen:

Any idea of what causes that?