nerfstudio-project / nerfstudio

A collaboration friendly studio for NeRFs
https://docs.nerf.studio
Apache License 2.0
9.55k stars 1.3k forks source link

Gaussian Splatting Viewer not working on Colab #2688

Open dlazares opened 11 months ago

dlazares commented 11 months ago

Describe the bug The beta gaussian splatting viewer is not working with the colab.

To Reproduce Steps to reproduce the behavior:

  1. Go to the repo and click the Colab button
  2. Change nerfacto to gaussian-splatting
  3. Run all the colab steps
  4. See Alternating Render connected and disconnected error.

Expected behavior Should see gaussian splat training

Screenshots


Viewer Beta at: http://0.0.0.0:7007/
(viser) Connection opened (309, 298 total), 1381 persistent messages connection handler failed Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/websockets/legacy/server.py", line 236, in handler await self.ws_handler(self) File "/usr/local/lib/python3.10/dist-packages/viser/infra/_infra.py", line 275, in serve await asyncio.gather( File "/usr/local/lib/python3.10/dist-packages/viser/infra/_infra.py", line 436, in _consumer message = message_class.deserialize(raw) File "/usr/local/lib/python3.10/dist-packages/viser/infra/_messages.py", line 99, in deserialize message_type = cls._subclass_from_type_string()[cast(str, mapping.pop("

brentyi commented 11 months ago

I think the problem here is the same as https://github.com/nerfstudio-project/viser/issues/148; the Colab by default seems like it opens a browser window with a viewer.nerf.studio URL, which isn't compatible with the beta viewer.

Untested, but I think things should work if we replace this snippet in the Colab:

with open('url.txt') as f:
  lines = f.readlines()
websocket_url = lines[0].split(": ")[1].strip().replace("https", "wss")
# from nerfstudio.utils.io import load_from_json
# from pathlib import Path
# json_filename = "nerfstudio/nerfstudio/viewer/app/package.json"
# version = load_from_json(Path(json_filename))["version"]
url = f"https://viewer.nerf.studio/?websocket_url={websocket_url}"
print(url)
print("You may need to click Refresh Page after you start training!")
from IPython import display
display.IFrame(src=url, height=800, width="100%")

with just:

with open('url.txt') as f:
  lines = f.readlines()
url = lines[0].split(": ")[1].strip()
print(url)
from IPython import display
display.IFrame(src=url, height=800, width="100%")