Windows 10. Crashes when you play the example scene

[andybak]

Error dialog is mostly empty:

Last section of the logs doesn't shed much light.

` 11:43:57 PROGRESS [=======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] 100% (50/50) 0s/0s 11:43:57 SUCCESS Loaded 50 images after 0s 11:43:57 INFO cam_aabb=[min=[1.0229,-1.33309,-0.378748], max=[2.46175,1.00721,1.41295]] 11:43:57 INFO Loading network config from: C:\Users\andyb\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\Instant Neural Graphics Primitives\data\nerf\fox\base.msgpack Refresh: detecting if any assets need to be imported or removed ...

Refresh Completed time: 0.383449s Asset Scan time: 0.343406s Asset Hashing: 0.000000s [0 B, 0.000000 mb/s] Asset Import (Scripting) time: 0.000000s (count: 0) Post Processs Assets (Scripting) time: 0.000000s Asset Rehashing: 0.000000s [0 B, 0.000000 mb/s] Asset Import (Non Scripting) time: 0.000000s (count: 0) Post Process Assets (Non Scripting) time: 0.000000s Dependent Assets to Import Queue time: 0.000000s`

[keli95566]

Hi! Thank you for the feedback!

Was the model trained with the instant-ngp instance from this repo?

[andybak]

Ah no. Is that a requirement?

Message ID: @.***>

[keli95566]

This could make a difference, as different instant-ngp versions might generate/load the model differently.

Let me know if this fixes the issue.

[andybak]

What file is the model written to? Running testbed.exe doesn't seem to write any training data by default. I can generate a snapshot (base.msgpack) - is that what is read in by the Unity plugin?

[keli95566]

Yes, the native plugin read the base.msgpack as the instannt-ngp.

• Make sure that in the same folder, there is base.msgpack, transform.json, and the images folder :)

Thank you for your contributions, once everything is working for you, we will update the readme to clarify this for other users.

[andybak]

The biggest help would be more informative error messages. Especially if they relate to paths or files not being found.

[andybak]

Also - providing builds of the dlls would save a lot of time for people - that's probably the most brittle part of the whole process.

[andybak]

Progress report. Now I've got a base.msgpack I'm no longer getting crashes. I've got nothing in the headset but Unity is showing the following:

It's tracking my head but only reading about 1 frame every 3 seconds. The low framerate might be the reason it's not showing in the headset. (I'm running a mobile 3080 btw)

[keli95566]

The stuff shown in Unity is correct, the initial view is somehow in the middle of the volume, you could use the controller to navigate around the scene. May I ask which VR headset you used? We tested this on an Oculus Quest 2, and it requires SteamVR + Open VR.

[TheTimmy]

Hi there,

How is your frame rate when building the Nerf Model? Maybe there are still some problems with wrong or missing dependencies.

[andybak]

OK. It's working. I had to drop width and height down to about 250x250.

I only get an image in a square filling the front 30% of my vision whatever the width and height. I presume that's a current technical limitation?

Stereoscopy seems a bit strange but that could be just the noisy data and the low resolution.

[keli95566]

Good to hear! Thank you for all the feedback.

For benchmarking, we developed it on an Alienware laptop with RTX 2080. With DLSS support, the fox scene should run at stable 20-30 fps at 1000*1000 per eye resolution.

My colleague will help look into what might be the performance problem.

[keli95566]

The FoV can be adjusted with the scale of the image planes, we will add a script to update this automatically.

[TheTimmy]

Hey,

I tried to reproduce your problem, but without luck. Can you verify that you compiled the library with Vulkan and DLSS enabled or post the CMake output when you configure the project.

[YenYaoHan]

Hi there, have same issuse when i run the fox demo base.msgpack file... the unity will crash. anyone can help? ^^"

[TheTimmy]

Hi,

have rebuild the scene with the provided testbed implementation trough:

build\testbed.exe --scene ..\data\nerf\fox

Otherwise, there is currently a bug that results in a crash if a none compatible network is loaded.

[YenYaoHan]

yeah, i think i already did, any steps i miss?

[TheTimmy]

Ok that looks good. You mentioned in #6 that you could not find the ngp_shared.dll. Did you manage to compile it? Otherwise Unity will crash due to the dll missing to run the application.

[YenYaoHan]

Yes, when i use the intant-ngp which in your repo, the dll file will build sucessfully. I put them in to unity project already, and also the nerf_path.

[keli95566]

@YenYaoHan Would you mind sharing the unity editor crash log here so we could take a further look at what might be the problem? The crash log can be found : C:\Users[yourusername]\AppData\Local\Unity\Editor\Editor. log on Windows. Thank you!

[YenYaoHan]

Here is the log file , please check. Thanks!

Editor.log

[keli95566]

@YenYaoHan

I took a look at the log, it seems that there are two issues:

1. There is no VR headset connected to the PC. Therefore OpenVR was not initiated. But this won't lead to an editor crash, the scene would just return black screen. We will push a commit that adds keyboard control soon so that users without a VR headset can also use this plugin in Unity.

2. In line789 of the log, Loading network config from: D:\CV_Workspace\immersive-ngp\instant-ngp\data\nerf\fox\base, the path to the model seems wrong, should be Loading network config from: D:\CV_Workspace\immersive-ngp\instant-ngp\data\nerf\fox\base.msgpack. Could you check in your transforms.json file if the path to the model is written correctly? This would lead to a crash of the editor. We will soon add a warning script to handle this so it won't lead to a crash.

If 2 doesn't fix the issue, it could be a GPU driver problem. Perhaps try upgrading the Nvidia graphics card driver.

With regard to performance, we highly recommend upgrading the Nvidia graphics driver and downloading the Vulkan SDK for DLSS support. For most GPUs, only with DLSS support can reach good resolution + framerate for VR applications.

I hope this help and thank you very much for your contribution!

[YenYaoHan]

Hi, BIG Thanks for your works, Through a day, I think I already can run your repo, and run nerf model which made by myself. Just like you said, I missed the VR device steps, maybe is beacause I didn't think that is a problem, so i was skip that. Beacause I thought it can import the nerf model(base.msgpack)and rendered by Unity, but it seem more like "remote rendering" function to let instant-ngp render in background progress.

Agree with you about the PC mode without VR device is nessessary. Very thank about your works, and hope all is well for you and @TheTimmy

Cheer! :)