Open mark2142 opened 7 months ago
aras-p
commented
7 months ago I know there's an older version of the sort in the 0.7 version of the project but I haven't been able to fix using that
So you mean the older version also has the sorting issue?
What GPU is in your Lenovo machine?
mark2142
commented
7 months ago Not sure if the old version has the sorting issue. And I can't find the specs for the non pc-edition of the A3 glasses same issue on quest 3. The function CSCalcDistances is returning a large number of zeros on device. And each time the sort is run the number of zeroes is more. I think the sort is failing due to this distance calculation failing. Could there be an issue with FloatToSortableUint on device? What log can I give you to help. I can trace the values out at different stages of the sorting process but I feel that this distance calculation could be the issue. Is it possible the splats are being deactivated due to an incorrect distance calculation?
aras-p
commented
7 months ago Hmm yeah that is very curious. Apparently something goes wrong somewhere, but kinda hard to say where or why. Something in the GPU driver not quite liking or mishandling code that comes out of FloatToSortableUint? No idea :/
b0nes164
commented
6 months ago I'll open a PR to push the latest version of my sort some time this week, but the only bug I am aware of on the version here is a very rare TDR crash on lower end devices due to cache thrashing.
There could be a bug that is not being caught being by the current testing spread, as I have not tested on Qualcomm, and Qualcomm is wave size 128. No idea about Vulkan, as I've only tested on DX12, though AFAIK Vulkan should be more capable than DX12.
Could you also confirm if the older sorting version fixes the problem?
b0nes164
commented
6 months ago Hi @aras-p, I just want to give a roadmap for where I want to go with this PR and how long it might take.
It's very likely that there is a bug on Qualcomm GPUs as I mentioned above, I have not tested on Qualcomm. I have a Qualcomm GPU on the way which should arive sometime this weekend-ish, and so I expect to begin debugging then.
However, bugs aside, I had several things planned that I wanted to push and I figure this would be a good opportunity to roll them all together into one PR:
I expect this to take maybe 2 weeks. If you have any issues please let me know. 👍
aras-p
commented
6 months ago @b0nes164 much thanks for the continued interest in pushing portable GPU sorting state of the art!
That said, I'd like to avoid external (non-unity) package dependencies in this project. I could perhaps do the "vendoring" approach of like periodically literally copying parts of your GPUSorting repository into this one (with proper attribution etc.), but actually depending on the package directly I'm not too keen on.
Maybe this could be added an "experimental use-at-your own risk" option, disabled by default?
Possibly!
ZaneZee
commented
6 months ago Hey there, I'm running into the same issue here as you are when trying to build to the Quest 3. The gaussians seem to be popping in and out of existence, but only when building to the Quest. This popping is slowed down by the "Sort Nth Frame" parameter and does not happen when stopping the sorting code from running. This is using Vulkan. Any help from anyone that has solved this would be amazing, @b0nes164 if you have a solution that worked for you I'd love to hear it! Happy to validate any ideas if you don't have a Quest 3 available. I will update this if I find a fix!
b0nes164
commented
6 months ago Hi @ZaneZee.
So the laptop I bought to test the sorting on Qualcomm did not have the capabilities to even run the sort (boo). See https://github.com/b0nes164/GPUSorting/issues/3#issuecomment-2083490992.
However, if you are willing, I can write a debug version of the sort for you to run on your Quest in Unity, and then you can send the results back to me. Keep in mind this will be a long process that could take several days to finish.
Based on the video, only a small part of the sort is breaking as opposed to https://github.com/aras-p/UnityGaussianSplatting/pull/82#issuecomment-1848441088, but it's unclear exactly what's going wrong.
ZaneZee
commented
6 months ago Hey, @b0nes164 thanks for the quick response! I am absolutely willing to help however I can. Let me know what you'd like me to test and I'm happy to build it to my Quest 3 and give you the results. Totally understand the back and forth will be a process! From what I've already tested it seems to start with a denser blob of gaussians when first loading the scene, then quickly devolving into this blob of flashing gaussians. I can also provide an example of what this looks like if you think it would be helpful. Thanks again, looking forward to testing whatever ideas you've got!
b0nes164
commented
6 months ago @ZaneZee Sweet, I should have a debugger up by tonight.
b0nes164
commented
6 months ago @ZaneZee Alright. The debugging procedure is as follows:
Many thanks!
ZaneZee
commented
6 months ago @b0nes164 Amazing I will go through this when I can today and send back the results! Thanks!
ZaneZee
commented
6 months ago @b0nes164 Here are the logs, let me know if there is any issue with these or if you need more info! Happy to help where I can. Should be able to get back to you pretty quickly now if you need anything else! DebugLogs.zip
mark2142
commented
6 months ago I have made the logs for the phone used to run the lonovo A3. It is a mot motorola_edge_plus_2022_lenovo_A3_Android13.zip orola edge plus 2022. It has the same problem as quest 3 and the video in an earlier post shows.
b0nes164
commented
6 months ago @ZaneZee Do you have discord? If you do, add me: throw_away_1234. I don't want to clog up this issue thread anymore than we already have.
@mark2142 I appreciate the help, but I don't need any testing from you at the moment. Also you did not set up the tests correctly (probably did not attach the compute shader) as the data collected is unintialized memory values.
mark2142
commented
5 months ago @ZaneZee and @b0nes164 . What is the progress guys? Do you want more help? Do you have any more information on the problem?
b0nes164
commented
5 months ago Apologies, but I've been extremely busy with other work at the moment. There is definitely something breaking with the sort, but I haven't been able to replicate the issue in isolation. It's definitely fixable and is still on my to do list, but finding time is the issue at the moment. Thanks again to @ZaneZee for testing and their patience.
mark2142
commented
5 months ago Don't know why but another project based on this project seems to work. https://github.com/ptc-lexvandersluijs/Unity3DGS_VR
aras-p
commented
5 months ago Don't know why but another project based on this project seems to work. https://github.com/ptc-lexvandersluijs/Unity3DGS_VR
From the looks of it, it is based on much older version of this same project, where the GPU sorting routine was based on FidelityFX radix sort. That sort is several times slower, but apparently does not have the bug/issue with some GPUs.
b0nes164
commented
5 months ago Just to give a quick update, there are definitely transpilation issues going on. For whatever reason, using WaveGetLaneCount() or WaveGetLaneIndex() as a predicate for large ternary operations causes incorrect behavior for a single lane on the Quest 3 in Vulkan. See lines L189-L191, L283-285, and L471-472, and note that no race conditions are possible at any of these locations. Simply breaking the operation into multiple lines produced correct behavior.
While changing these lines was enough for the sort to work correctly in isolated single runs, it still fails when integrated with the splatting routine, just less than before.
More testing will be required to figure what else is breaking and what else the transpilation doesn't like.
ZaneZee
commented
5 months ago Hey @b0nes164, I sent you another message with some more info on discord. Let me know if you have time to continue trying to solve the issue here! Happy to help in any way I can still as well. Thanks for taking the time with what you've already looked into also, at the very least it is one step closer to being resolved. Looking forward to hearing from you!
busiyg
commented
3 months ago May I ask if there has been any progress on running it on Android devices? I use Vulkan in Qualcomm Snapdragon 870 Device，Android 14. nothing showed，and i get error in device：SplatUtilities.compute: Kernel at index (2) is invalid
busiyg
commented
2 months ago Hello, I'm trying to deploy the project to Android devices. Currently, I'm encountering some issues. I'm using Vulkan, and I have two Android devices that are exhibiting different abnormal behaviors.
1.The first device is an Android tablet running Android 14 with Qualcomm Snapdragon 870. The project runs on this device without any errors, but nothing appears on the screen. I've checked some values at the end of the GaussianSplatRenderer.SortPoints function.
2.I installed the same APK on the Quest Pro, and while there is content displayed, it doesn't look right. Additionally, the ellipsoids decrease as the viewing angle changes, but if the viewing angle remains stationary, they do not decrease.
https://github.com/user-attachments/assets/6a7fa64a-f736-4ce3-9afb-808012a1ff51
busiyg
commented
2 months ago I can provide more debugging information if needed. At the moment, I suspect there might be some issues with GPUSorting, but I'm not sure what to do about it. @b0nes164
b0nes164
commented
2 months ago Fix still pending. See Different Android devices produce different results.
AndreCorreaSantos
commented
2 months ago Hi everyone, I have a question and I'm not sure if this is the right place to ask. I want to set an integer property in the RenderGaussianSplats shader. I've been trying to set it using Material Property Blocks, but I haven't had any success so far. What is the common way to set properties in this shader? I simply want to pass a position and radius of a sphere I have in the scene to try and compute which splats end up inside it and then apply some custom shaders to these splats. It should be very simple but i'm struggling, any help would be greatly appreciated, thanks in advance and sorry for posting in the wrong section.
edhyah
commented
2 months ago @mark2142 did you manage to get it working with the older version of the sort (v0.7)? for me, i just see a black screen
AFMagnon
commented
1 week ago The same problem I am facing. https://github.com/aras-p/UnityGaussianSplatting/issues/112#issuecomment-2282412914
Build and installation were succeeded. Howevever, nothing appeared in the application. Not to mention, normal 3D-objects were displayed.
The way of setting for the Build went wrong??
First I want to say thank you to the author for his great work and effort. I am using Vulkan. I am using a Lenovo A3 and my brother is using a quest 3 and we have the same problem. I can see the splats on device. However the splats seem to quickly go from unsorted but visible to a very sparse set of shimmering splats. When I set the number of frames between sort to a high number it takes longer for the splats to fade away. If I don't sort at all, All splats are there but out of order and don't change. So the sort is the problem. Using Vulkan it does not complain about the wave intrinsics. So as far as I can tell the sort is trashing my splats. I know there's an older version of the sort in the 0.7 version of the project but I haven't been able to fix using that. Anybody have a fix to the sort algorithm or different GPU sort or any other ideas on how to fix this?