search

microsoft / HoloJS

Provides a framework for creating holographic apps using JavaScript and WebGL.
MIT License
1.19k stars 114 forks source link

Multiple shader support #24

Closed sjando closed 7 years ago

sjando commented 7 years ago

Currently the automatic stereo rendering and projection used by HoloJS only works with one active shader at a time. Ideally HoloJS should be able to support applications using any number of shaders.

Perhaps instead of automating this process, all the relevant information (pose, left/right view matrices, left/right projection matrices) could be exposed to javascript allowing all shaders to utilise it. WebVR does something very similar to this. In fact, maybe it makes sense to expose the information via a WebVR-compatible API? The automated solution could be still be optionally available for ease-of-use in smaller applications.

sjando commented 7 years ago

I've got a working example of multiple shaders running with three.js over at https://github.com/sjando/HoloJS/tree/three-multi-shader. The 'advanced drawing' mode of ANGLE ms-holographic-experimental is used and a three.js plugin handles all the instanced stereo rendering on the WebGL side. This required some small changes to ANGLE to expose all the required matrices. These changes also fix #25 (three.js culling issue). Could do a PR if this issue hasn't already been tackled (or would it be better to have any required changes merged into https://github.com/Microsoft/angle/tree/ms-holographic-experimental first?)

Almost-Done commented 7 years ago

@sjando Sorry for my incredibly late reply; I started fixing the culling issue and as I was referencing the original issue I saw your reply above.

You made amazing progress with ThreeJS rendering support; it would be great if you want to do a PR and merge your work.

We can merge changes to angle/ms-holographic-experimental as needed. I noticed you removed the angle submodule in your fork. Would it be OK to keep the submodule, merge changes to ANGLE and move the angle submodule pointer forward?

trusktr commented 7 years ago

Seems I posted a related issue: https://github.com/Microsoft/HoloJS/issues/62

@sjando Does your stuff still work? Can you list some steps on how to try your mod?

Almost-Done commented 7 years ago

Good news everyone! I believe we have a simple solution for this issue.

We found a bug in angle-holographic involving the management of auto-stereo shaders. All shaders were converted to stereo, but the holographic rendering parameters are only updated for one of them - the last shader used in the previous frame. The fix is to update the shader's holographic parameters each time the program is changed, not just once once per frame on reset.

Please give this a try in the develop/multiple-shaders branch and let me know if it fixes this issue.

Reda-S commented 7 years ago

Hi Cristi,

Multishader support is working! Keep up the good work.

& very nice work on the web-ar protocol based viewer (wondering why not just make it part of the HoloLens edge browser, users wouldn't need to install any additional app they just browse to a web-ar address...)

Almost-Done commented 7 years ago

Thank you for verifying the fix. I can't believe I get to close this issue.

About the web-ar sample, I'm sorry but can't comment on any browser plans.

sjando commented 7 years ago

Excellent stuff! Never found time to update my old pull request but this makes a lot of the ugly stuff in there unnecessary. Where can we find the updated angle-holographic code? Would be great if we can still expose a projection matrix to make culling work in e.g. three.js but retain the auto-stereo functionality.

@Almost-Done regarding your old comment about removing the angle submodule; if we are using prebuilt angle binaries now do we still need it? Right now the binaries and the submodule aren't even in sync anymore are they?

Almost-Done commented 7 years ago

@sjando ANGLE is not a submodule of HoloJS anymore. To expedite the required ANGLE fixes, I forked ANGLE; the fixes are in https://github.com/Almost-Done/angle/tree/crispet/update-multiple-shaders

I was about to propose that you simplify your pull request to just fixing culling and we can take the required ANGLE changes to the forked branch for now. I'm working with ANGLE folks to merge these fixes at least into ms-experimental-holographic.

sjando commented 7 years ago

Submodule is still is in at the moment and relied upon for header files: https://github.com/Microsoft/HoloJS/blob/master/HoloJS/HoloJsHost/HoloJsHost.vcxproj#L127

Submodule hasn't been updated to point at your branch though.

I'll see if I can get angle changes into the forked branch and create a new pull request here for culling only

Almost-Done commented 7 years ago

You are right, thank you for pointing out. I removed it from the solution but it's still a submodule and required for headers.

Almost-Done commented 7 years ago

@sjando I created a pull request to update the submodule to point to the holojs/master branch.

Probably these commands will be required: git submodule sync git submodule update --init --recursive --remote