search

MarekKowalski / LiveScan3D-Hololens

A Unity application for receiving and rendering point clouds on Hololens and other devices.
MIT License
570 stars 144 forks source link

Release for Hololens 2? #26

Open fberna opened 3 years ago

fberna commented 3 years ago

Hi Marek,

any news about the release for HL2? Hope everything is fine.

FB

Originally posted by @fberna in https://github.com/MarekKowalski/LiveScan3D-Hololens/issues/20#issuecomment-801742143

MarekKowalski commented 3 years ago

Hi Fabio, thanks for asking. I did some work on it a few days ago and hope to get a chance to do a little more on Sunday or Monday. If I don't write here till Tuesday, please ping me.

MarekKowalski commented 3 years ago

I pushed an initial version for HL2 to a new branch called hololens_2. At the moment it does not allow for moving the point cloud around and it requires the server's IP address to be hardcoded in line 21 of KeyboardInput.cs. Also make sure to compile the UWP project using ARM64.

If anyone gets a chance to try it, please let me know.

Thanks,

Marek

fberna commented 3 years ago

Hi Marek, thank you for your release.

I had the opportunity to test it this morning and I faced some troubles. When I play the app directly from Unity the connection seems to work. During compiling it in ARM64, I got an error about ucrtbase.dll ("RangeChecks instrumentation code detected an out of range array access"). Running the app on HL2, nothing happens only turning spheres for 30 sec then closes itself.

Do you think it could be a network problem? or Is it related to the compile process?

Thank you so much for your support!

FB

MarekKowalski commented 3 years ago

Hmm, interesting.. Have you tried rebuilding the NetworkCommunication project? If not please try that and make sure to use ARM64. Maybe there is something wrong with the checked-in DLL. If you do that make sure to re-generate the Visual Studio project from Unity.

TranDucHuyVnuHcmUs commented 3 years ago

I pushed an initial version for HL2 to a new branch called hololens_2. At the moment it does not allow for moving the point cloud around and it requires the server's IP address to be hardcoded in line 21 of KeyboardInput.cs. Also make sure to compile the UWP project using ARM64.

If anyone gets a chance to try it, please let me know.

Thanks,

Marek

Just got it working on Hololens 2. Rebuilding NetworkCommunication project first, using ARM64. Thanks Marek!

regexboi commented 3 years ago

Hi Marek. I also got the project working by rebuilding NetworkCommunication project, and then building HololensReceiver in Unity before deploying the resulting .sln via Visual Studio (was not able to rebuild the .sln directly in Visual Studio, although it seemed to be due to missing dependencies on my end).

Anyway, once built, I get a bunch of white triangles overlaid on the environment. Any idea what could be causing this?

regexboi commented 3 years ago

https://ibb.co/0Jsfj7t

MarekKowalski commented 3 years ago

Hi! These white triangles look like the spatial mapping mesh. You can change the mesh visibility in the settngs of the MixedRealityToolkit game object. image

KaiKruschel commented 3 years ago

Hi Marek, we got this to work fine on our Hololens 2 a couple of weeks ago however now I can't reproduce it for the life of me.

At the moment the Hololens does not seem to place the Point cloud in the world. Rather it is attached to the hololens. Since it spawned behind the camera and then moves with the camera to alway stay behind it, it took quite some time to find this out. After I moved the pointcloudRenderer in the scene to be further in front of the camera, I could see the point cloud being rendered but very choppy and uneven and also moving with the head.

I also had a couple of crashes with the ucrtbase.dll error. However it seems now, that the problem might have been network related. This seems to happen if the server is not running or if no connection can be established with it.

I then had the same issue as Mdec-eng with the spatial mapping mesh showing. I switched to the defaultHololens2ConfigurationProfile in the Toolkit GameObject and that went away.

Now the only problem is the point cloud not showing in a fixed position but being attached to the headset and also the livestreaming being really choppy and uneven. Both of these things did not happen, when we first got it to work. Do you have any idea what might be causing these problems?

Thanks, Kai

MarekKowalski commented 2 years ago

Thanks for writing. Wow, that sound weird. The choppiness could be due to the bandwidth of the connection between the computer and the HL2. If the bandwidth is small you may want to limit the reconstruction volume in the settings of the LiveScanServer as this will reduce the required bandwidth.

I'm not sure where the issue with point cloud moving together with device might come from. One thing I'd do to debug this would be to place an object (say a cube) in the scene at the same place in hierarchy as the point cloud and if it moves with the headset too. If it does, then it's a problem with scene settings, if it stays in place, then the problem is specific to the point cloud.

KaiKruschel commented 2 years ago

I will try restricting the reconstruction volume. Thanks for the tip.

I also had the idea with the cube and it also doesn't stay in place so it seems to be something with the scene settings. The weird thing is that this happens with a clean clone from the repo. I setup up a seperate new hololens project from scratch and that worked fine so I copied over the PointCloudRenderer, Receiver and the material and this setup works for now but I have no idea what would cause this problem in the first place. Especially since it has worked before on the same device without issue. Oh well.

Thanks for the help (and the software)

cevo2022 commented 1 year ago

Hi, Kaikruschel I am now on the same journey like you before to streaming the point cloud of Azure kinect DK to Hololens2, do you mind to share the Repo of this project?