Open smtabatabaie opened 10 months ago
Oh, OpenGL or OpenGL ES is definitely not going to be supported. If Quest supports Vulkan, try using that instead?
nope, black screen on quest last time I tested with vulkan in URP... I was able to run on PC in VR with Quest in QuestLink and OpenXR. Will retry on Quest with Vulkan and OpenXR see if it is better...
i was aiming for: rendering gaussian splats on Native Quest optimizing and squeezing in as much fps as possible having atleast 15 frames per second
sadly with Vulkan and OpenXR it's working at around 20 fps which is great but the only thing that is rendering is the color of the skybox.. for now there's only one option which is the gaussian splatting viewer implementation in A-Frame, but that's working on WebXR at 2 fps.. and there's no room for optimization like unity has. Thank you for your existing AMAZING work, but the best use-case for this tech is VR (standalone of course) even if it only reaches 20 fps on Quest 2 Devices (they're outdated now so it's no surprise). for now, The Unity Gaussian Splatting plugin is useless for just that, unfortunately, hopefully you'll see this message @aras-p and realize that Native Quest support in this plugin should be priority number 1! Good Luck ! God Bless you <3
sadly with Vulkan and OpenXR it's working at around 20 fps which is great but the only thing that is rendering is the color of the skybox..
Is that some known issue with Unity or OpenXR Vulkan support? Maybe it's fixed in a more recent Unity version by chance (e.g. latest 2023 alpha/beta)?
see this message @aras-p and realize that Native Quest support in this plugin should be priority number 1!
I do see this message, yes. I have exactly zero VR devices around, so adding any sort of VR support is quite a bit outside of my reach.
Shader error in 'Gaussian Splatting/Debug/Render Boxes': no matching function for call to 'asfloat' at Assets/GSPLAT/GaussianSplatting/Shaders/GaussianSplatting.hlsl(527) (on vulkan)
@aras-p
i don't know if this is why it's not rendering, i updated to 2023.1.16f1
Shader error in 'Gaussian Splatting/Debug/Render Boxes': no matching function for call to 'asfloat' at Assets/GSPLAT/GaussianSplatting/Shaders/GaussianSplatting.hlsl(527) (on vulkan)
Exactly what I'm getting when I try to run on Quest 2
@electrum-bowie I edited your post to make it shorter. And yes, that's exactly the error mentioned at the beginning of this issue. But until either someone contributes a fix, or I get a VR device (which is unlikely to happen), it will stay like that.
Shader error in 'Gaussian Splatting/Debug/Render Boxes': no matching function for call to 'asfloat' at Assets/GSPLAT/GaussianSplatting/Shaders/GaussianSplatting.hlsl(527) (on vulkan)
Exactly what I'm getting when I try to run on Quest 2
The same issue appears when targeting any Android device. What makes me curious is that by insisting it compiles and export apk but then, of course, just a black screen. When trying to compile version 0.2.0 it works, sort of. It display only some splats with wrong sizes and positions and not all of them and it's very heavy task for android devices, even with powerful ones. By selecting debug points instead of splats it becomes "usable" but again not all points are rendered, only a few of them. Maybe it was partially working because there was also another code for sorting the splats. I also tried exporting for iOS but only 0.6.0 and as expected I had only black screen but on conosle debugger it was reporting problems related to FfxParallelSort.
With the latest code v0.7.0, instead of producing the abovementioned error, sorting kernel compatibility check won't pass (here in the code, only m_KernelSum
is supported) when targeting to Android.
Edit: The above error only occurs on macOS. On Windows it's the same "no matching function for call" error. I tried to remove https://github.com/aras-p/UnityGaussianSplatting/blob/fc97e079b04f6b1701605f9f019411ce61d49c9a/package/Shaders/GaussianSplatting.hlsl#L525-L542 (where the asfloat() error occurred) and use "High" quality assets. The compilation passed but it's still a black screen when running on an Android phone.
I tried to dig into the Android logcat and here are the outputs:
It seems to be related to the unsupported texture format on mobile (related to #44? but I'm using the "High" quality). I'm using the Vulkan backend.
In case it helps in gathering information, I was able to verify that using the Vulkan rendering within Spaces SDK on a VR device is able to render Gaussian Splats on one eye a week or so ago.
In case it helps in gathering information, I was able to verify that using the Vulkan rendering within Spaces SDK on a VR device is able to render Gaussian Splats on one eye a week or so ago.
Thanks @jacemiller for the info! Are you referring to native rendering on the VR device or PCVR? Which device are you using? How do you manage to get it run on an Android device?
@JiamingSuen
I got the project rendering natively on the Android-based VR devices (Lenovo VRX and Motorola edge+/Lenovo A3). I was also using Spaces SDK, though really only for the spatial navigation of virtual camera pose and standard VR camera rig compatible with the devices. One other comment is that the splat content was only visible in one of the two eye views - the other rendered completely black.
I only looked into the project briefly, so my comments may not be related to the latest revision. The version I had working was based off a commit that did include the menu option Tools->"Gaussian Splats"->"Create GaussianSplatAsset" before it was removed and subsequently replaced (originally cloned approximately around September 23rd).
There seemed to be some issues with rendering using OpenGL. Two items that seemed to be key aspects were:
So, maybe there is some lack of "Wave*" function support on the Android-based devices I tried that is assumed in the computer shader? There is a "#pragma require wavebasic" line at the top of the FfxParallelSort file.
In case it helps in gathering information, I was able to verify that using the Vulkan rendering within Spaces SDK on a VR device is able to render Gaussian Splats on one eye a week or so ago.
Even when using Windows as target device and using the standard OpenXR features, the camera is only rendering 1 eye. So this does not seem to be an android issue. Anyone managed to solve this targetting windows?
Even when using Windows as target device and using the standard OpenXR features, the camera is only rendering 1 eye
Yes, various forms of VR have various issues, that are under #17
using the standard OpenXR features, the camera is only rendering 1 eye @Zulex @jacemiller
I fixed this by changing 'Render Mode' to Multi-pass under Project Settings > XR Plug-in Management > OpenXR.
Ok I got it working on Vulkan. What you need is: 1) Use multi-pass. 2) The discard trick does not compile well, for now just make the alpha = 0
if (behindCam)
{
o.col = half4(0,0,0,0);
//o.vertex = asfloat(0x7fc00000); // NaN discards the primitive
}
3) Replace every asfloat
in GaussianSplatting.hlsl
with:
float asfloat2(uint u)
{
double temp = asdouble(uint(0), u);
return float(temp);
}
Except in the sh calculation at the end where you can just skip using asfloat() at the end, for example::
s.sh.sh1 = lerp(shMin, shMax, s.sh.sh1 );
@pablodawson while that might work as a workaround, it does read as a very large "oof!". Like, it should not actually work :/
Big oof indeed, no idea why it's working now
i was aiming for: rendering gaussian splats on Native Quest optimizing and squeezing in as much fps as possible having atleast 15 frames per second
sadly with Vulkan and OpenXR it's working at around 20 fps which is great but the only thing that is rendering is the color of the skybox.. for now there's only one option which is the gaussian splatting viewer implementation in A-Frame, but that's working on WebXR at 2 fps.. and there's no room for optimization like unity has. Thank you for your existing AMAZING work, but the best use-case for this tech is VR (standalone of course) even if it only reaches 20 fps on Quest 2 Devices (they're outdated now so it's no surprise). for now, The Unity Gaussian Splatting plugin is useless for just that, unfortunately, hopefully you'll see this message @aras-p and realize that Native Quest support in this plugin should be priority number 1! Good Luck ! God Bless you <3
Hello, how to run 3d Gauss on unity Android?
Hi, Thanks for the awesome project. I could run it via OpenXR and Quest Link (tethered to my desktop computer with a 3090), but when I try to run it on Quest standalone and build an apk, I get 50 errors regarding the shader files like this:
error in 'Gaussian Splatting/Debug/Render Boxes': 'asfloat': no matching 1 parameter intrinsic function; Possible intrinsic functions are: asfloat(float|half|int|uint) at Assets/GaussianSplatting/Shaders/GaussianSplatting.hlsl(469) (on gles3)
Has anyone succeeded in running Gaussian Splatting on Quest standalone? Thanks