Mac Installation Instructions?

sudoku-lord commented 1 year ago

Hi! I saw on this link that NeRFStudio is now compatible with MacBooks which have M1/M2 chips. I'm wondering if the instructions for installation are the same as the PC ones, or whether there's a separate set of instructions for Mac installation available somewhere else. Additionally, are there Docker images for Mac?

Thank you so much!

tancik commented 1 year ago

For mac, just run pip install nerfstudio, skip the tiny-cuda-nn and pytorch dependency steps.

LuyuanPeng commented 1 year ago

For mac, just run pip install nerfstudio, skip the tiny-cuda-nn and pytorch dependency steps.

I have a M2 Mac on Ventura 13.4.1. I could not install nerfstudio using pip3 install nerfstudio in a clean conda environment. The output for this command is as below.

Collecting nerfstudio
  Obtaining dependency information for nerfstudio from https://files.pythonhosted.org/packages/2d/db/2f6697579b7813b5413118726f30cb6e3e6427580385943c797d9cbd02cb/nerfstudio-0.3.3-py3-none-any.whl.metadata
  Using cached nerfstudio-0.3.3-py3-none-any.whl.metadata (23 kB)
Collecting appdirs>=1.4 (from nerfstudio)
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting av>=9.2.0 (from nerfstudio)
  Using cached av-10.0.0-cp311-cp311-macosx_11_0_arm64.whl (19.6 MB)
Collecting cryptography>=38 (from nerfstudio)
  Obtaining dependency information for cryptography>=38 from https://files.pythonhosted.org/packages/91/68/5c33bb0115b3413a974dd4d23625b99ed22522582b513f82e93ce00f954c/cryptography-41.0.3-cp37-abi3-macosx_10_12_universal2.whl.metadata
  Using cached cryptography-41.0.3-cp37-abi3-macosx_10_12_universal2.whl.metadata (5.2 kB)
Collecting tyro>=0.5.3 (from nerfstudio)
  Obtaining dependency information for tyro>=0.5.3 from https://files.pythonhosted.org/packages/18/90/ec14750938d4d9f18f3165d9f89ef8ba359c15fcb2c7e0c08e54d8705cff/tyro-0.5.7-py3-none-any.whl.metadata
  Using cached tyro-0.5.7-py3-none-any.whl.metadata (6.4 kB)
Collecting gdown>=4.6.0 (from nerfstudio)
  Using cached gdown-4.7.1-py3-none-any.whl (15 kB)
Collecting ninja>=1.10 (from nerfstudio)
  Using cached ninja-1.11.1-py2.py3-none-macosx_10_9_universal2.macosx_10_9_x86_64.macosx_11_0_arm64.macosx_11_0_universal2.whl (270 kB)
Collecting h5py>=2.9.0 (from nerfstudio)
  Obtaining dependency information for h5py>=2.9.0 from https://files.pythonhosted.org/packages/37/82/859d9fc800a316e692bd040437d45613512b43d49fb0435c5bbab9e71511/h5py-3.9.0-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Using cached h5py-3.9.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.5 kB)
Collecting imageio>=2.21.1 (from nerfstudio)
  Obtaining dependency information for imageio>=2.21.1 from https://files.pythonhosted.org/packages/eb/21/662994d78d8623055f8ffa91838e28f04b2a34bd5d8d6dbc6c7573285ed6/imageio-2.31.3-py3-none-any.whl.metadata
  Using cached imageio-2.31.3-py3-none-any.whl.metadata (4.6 kB)
Collecting ipywidgets>=7.6 (from nerfstudio)
  Obtaining dependency information for ipywidgets>=7.6 from https://files.pythonhosted.org/packages/4a/0e/57ed498fafbc60419a9332d872e929879ceba2d73cb11d284d7112472b3e/ipywidgets-8.1.1-py3-none-any.whl.metadata
  Using cached ipywidgets-8.1.1-py3-none-any.whl.metadata (2.4 kB)
Collecting jaxtyping>=0.2.15 (from nerfstudio)
  Obtaining dependency information for jaxtyping>=0.2.15 from https://files.pythonhosted.org/packages/ee/a3/fea2d62ed8b055102d52a8c2df435eb9ca924ef3d65d7114308be0ded0a7/jaxtyping-0.2.21-py3-none-any.whl.metadata
  Using cached jaxtyping-0.2.21-py3-none-any.whl.metadata (6.0 kB)
Collecting jupyterlab>=3.3.4 (from nerfstudio)
  Obtaining dependency information for jupyterlab>=3.3.4 from https://files.pythonhosted.org/packages/3b/43/2368d8ffee6e33f282f548d42fa222bd385cc9f66545b260e7d08e90046b/jupyterlab-4.0.6-py3-none-any.whl.metadata
  Using cached jupyterlab-4.0.6-py3-none-any.whl.metadata (15 kB)
Requirement already satisfied: matplotlib>=3.5.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from nerfstudio) (3.7.2)
Collecting mediapy>=1.1.0 (from nerfstudio)
  Obtaining dependency information for mediapy>=1.1.0 from https://files.pythonhosted.org/packages/22/e8/63417295c027d2b4ee475798168e21b7f3265304e0ca77b9d65b5e8aff05/mediapy-1.1.9-py3-none-any.whl.metadata
  Using cached mediapy-1.1.9-py3-none-any.whl.metadata (4.8 kB)
Collecting msgpack>=1.0.4 (from nerfstudio)
  Using cached msgpack-1.0.5-cp311-cp311-macosx_11_0_arm64.whl (69 kB)
Collecting msgpack-numpy>=0.4.8 (from nerfstudio)
  Using cached msgpack_numpy-0.4.8-py2.py3-none-any.whl (6.9 kB)
Collecting nerfacc==0.5.2 (from nerfstudio)
  Using cached nerfacc-0.5.2-py3-none-any.whl (55 kB)
INFO: pip is looking at multiple versions of nerfstudio to determine which version is compatible with other requirements. This could take a while.
Collecting nerfstudio
  Obtaining dependency information for nerfstudio from https://files.pythonhosted.org/packages/44/6e/721838b5b4aee4f144d94f556532bcd94b0b59fb955c078857ae86e3c737/nerfstudio-0.3.2-py3-none-any.whl.metadata
  Using cached nerfstudio-0.3.2-py3-none-any.whl.metadata (22 kB)
  Obtaining dependency information for nerfstudio from https://files.pythonhosted.org/packages/28/e2/9042b7fa9fbccaccf2695fefa831c924222afd4e800789a8ab817cf76a71/nerfstudio-0.3.1-py3-none-any.whl.metadata
  Using cached nerfstudio-0.3.1-py3-none-any.whl.metadata (22 kB)
  Obtaining dependency information for nerfstudio from https://files.pythonhosted.org/packages/4a/d9/537f8fe6e6af1043697f2171d95d6ff9d35a62919f0a60e0a784bd991086/nerfstudio-0.3.0-py3-none-any.whl.metadata
  Using cached nerfstudio-0.3.0-py3-none-any.whl.metadata (23 kB)
  Using cached nerfstudio-0.2.2-py3-none-any.whl (437 kB)
Collecting functorch>=0.2.1 (from nerfstudio)
  Using cached functorch-2.0.0-py2.py3-none-any.whl (2.1 kB)
Collecting nerfstudio
  Using cached nerfstudio-0.2.1-py3-none-any.whl (425 kB)
Collecting nerfacc==0.3.5 (from nerfstudio)
  Using cached nerfacc-0.3.5-py3-none-any.whl (42 kB)
Collecting nerfstudio
  Using cached nerfstudio-0.2.0-py3-none-any.whl (423 kB)
  Using cached nerfstudio-0.1.19-py3-none-any.whl (385 kB)
INFO: pip is still looking at multiple versions of nerfstudio to determine which version is compatible with other requirements. This could take a while.
  Using cached nerfstudio-0.1.18-py3-none-any.whl (371 kB)
Collecting aiohttp>=3.8.1 (from nerfstudio)
  Obtaining dependency information for aiohttp>=3.8.1 from https://files.pythonhosted.org/packages/47/10/33abd984a476e314afdb4711fbd0aac1b25927676fa591445537da3aee98/aiohttp-3.8.5-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Using cached aiohttp-3.8.5-cp311-cp311-macosx_11_0_arm64.whl.metadata (7.7 kB)
Collecting aiortc>=1.3.2 (from nerfstudio)
  Using cached aiortc-1.5.0-cp311-cp311-macosx_11_0_arm64.whl (867 kB)
Collecting cryptography<39,>=38 (from nerfstudio)
  Using cached cryptography-38.0.4-cp36-abi3-macosx_10_10_universal2.whl (5.4 MB)
Collecting nerfacc==0.3.3 (from nerfstudio)
  Using cached nerfacc-0.3.3-py3-none-any.whl (45 kB)
Collecting nerfstudio
  Using cached nerfstudio-0.1.17-py3-none-any.whl (368 kB)
  Using cached nerfstudio-0.1.16-py3-none-any.whl (335 kB)
  Using cached nerfstudio-0.1.15-py3-none-any.whl (312 kB)
Collecting aiohttp==3.8.1 (from nerfstudio)
  Using cached aiohttp-3.8.1.tar.gz (7.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting aiortc==1.3.2 (from nerfstudio)
  Using cached aiortc-1.3.2.tar.gz (1.2 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting av==9.2.0 (from nerfstudio)
  Using cached av-9.2.0.tar.gz (2.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      pkg-config is required for building PyAV
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Any ideas how to solve this? Thanks.

tancik commented 1 year ago

Maybe this is helpful? https://stackoverflow.com/questions/30445428/cant-install-pyav-on-mac-pkg-config-is-required

sudoku-lord commented 1 year ago

@tancik

I'm actually getting another error when I try to use custom data. ns-train nerfacto --data output_data gives me:

AssertionError: Torch not compiled with CUDA enabled

Is there a workaround for this issue?

Edit to add: my Mac is also M2 on Ventura 13.4. I've previously had no success using NeRFStudio Google Colab, and don't have access to a PC, so I'm really hoping to get some clarity on how to run it on my Mac. Thank you so much!

jonahclarsen commented 1 year ago

For reference, I had the same pkg-config is required for building PyAV error when trying to pip install nerfstudio on my M2 MBA (MacOS 14.0). I tried pip install pkgconfig which executed successfully but got the same error. I then tried brew install pkgconfig which executed successfully but gave me a new error, also at the PyAV compilation step:

Collecting av==9.2.0 (from nerfstudio)
  Using cached av-9.2.0.tar.gz (2.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      Package libavformat was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavformat.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavformat' found
      Package libavcodec was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavcodec.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavcodec' found
      Package libavdevice was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavdevice.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavdevice' found
      Package libavutil was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavutil.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavutil' found
      Package libavfilter was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavfilter.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavfilter' found
      Package libswscale was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libswscale.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libswscale' found
      Package libswresample was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libswresample.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libswresample' found
      pkg-config could not find libraries ['avformat', 'avcodec', 'avdevice', 'avutil', 'avfilter', 'swscale', 'swresample']
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

No clue where to go from here. @LuyuanPeng did you ever find a solution? Any help would be appreciated.

LuyuanPeng commented 1 year ago

For reference, I had the same pkg-config is required for building PyAV error when trying to pip install nerfstudio on my M2 MBA (MacOS 14.0). I tried pip install pkgconfig which executed successfully but got the same error. I then tried brew install pkgconfig which executed successfully but gave me a new error, also at the PyAV compilation step:

Collecting av==9.2.0 (from nerfstudio)
  Using cached av-9.2.0.tar.gz (2.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      Package libavformat was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavformat.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavformat' found
      Package libavcodec was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavcodec.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavcodec' found
      Package libavdevice was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavdevice.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavdevice' found
      Package libavutil was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavutil.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavutil' found
      Package libavfilter was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libavfilter.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libavfilter' found
      Package libswscale was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libswscale.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libswscale' found
      Package libswresample was not found in the pkg-config search path.
      Perhaps you should add the directory containing `libswresample.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'libswresample' found
      pkg-config could not find libraries ['avformat', 'avcodec', 'avdevice', 'avutil', 'avfilter', 'swscale', 'swresample']
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

No clue where to go from here. @LuyuanPeng did you ever find a solution? Any help would be appreciated.

Yes, I managed to get it work by commenting out sections where pyav was required. For my program, nerfstudio works fine without using pyav. @jonahclarsen

jonahclarsen commented 1 year ago

Now I can get it to install, and using the training command from #2436 I got past a Torch not compiled with CUDA enabled error.

That command is: PYTORCH_ENABLE_MPS_FALLBACK=1 ns-train nerfacto --data data/nerfstudio/poster --machine.device-type mps --mixed-precision False

But then I got this error: User specified an unsupported autocast device_type 'mps'

Implementing the change to trainer.py from #2436 fixed that. But now I get this error, not sure what to do:

[NOTE] Not running eval iterations since only viewer is enabled.
Use --vis {wandb, tensorboard, viewer+wandb, viewer+tensorboard} to run with eval.
No Nerfstudio checkpoint to load, so training from scratch.
Disabled comet/tensorboard/wandb event writers
Process DataProcessor-2:
Traceback (most recent call last):
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/Users/me/Downloads/nerfstudio/nerfstudio/data/datamanagers/parallel_datamanager.py", line 117, in run
    ray_bundle: RayBundle = self.ray_generator(ray_indices)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/me/Downloads/nerfstudio/nerfstudio/model_components/ray_generators.py", line 52, in forward
    ray_bundle = self.cameras.generate_rays(
  File "/Users/me/Downloads/nerfstudio/nerfstudio/cameras/cameras.py", line 463, in generate_rays
    raybundle = cameras._generate_rays_from_coords(
  File "/Users/me/Downloads/nerfstudio/nerfstudio/cameras/cameras.py", line 646, in _generate_rays_from_coords
    coord_stack[coord_mask, :] = camera_utils.radial_and_tangential_undistort(
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/eval_frame.py", line 328, in _fn
    return fn(*args, **kwargs)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/eval_frame.py", line 490, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 641, in _convert_frame
    result = inner_convert(frame, cache_size, hooks, frame_state)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 133, in _fn
    return fn(*args, **kwargs)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 389, in _convert_frame_assert
    return _compile(
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 569, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/utils.py", line 189, in time_wrapper
    r = func(*args, **kwargs)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 491, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/bytecode_transformation.py", line 1028, in transform_code_object
    transformations(instructions, code_options)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 458, in transform
    tracer.run()
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2074, in run
    super().run()
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 724, in run
    and self.step()
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 688, in step
    getattr(self, inst.opname)(inst)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2162, in RETURN_VALUE
    self.output.compile_subgraph(
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/output_graph.py", line 833, in compile_subgraph
    self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/output_graph.py", line 957, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/utils.py", line 189, in time_wrapper
    r = func(*args, **kwargs)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/output_graph.py", line 1024, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/output_graph.py", line 1009, in call_user_compiler
    compiled_fn = compiler_fn(gm, self.example_inputs())
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/_dynamo/repro/after_dynamo.py", line 117, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/torch/__init__.py", line 1607, in __call__
    return self.compiler_fn(model_, inputs_, **self.kwargs)
torch._dynamo.exc.BackendCompilerFailed: backend='eager' raised:
TypeError: eager() got an unexpected keyword argument 'mode'

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True

Ok I actually managed to get past that by adding torch._dynamo.config.suppress_errors = True to line 462 of cameras.py in nerfstudio. Not sure what I'm giving up by doing that but it is training now.

Edit: never mind, I spoke too soon. It seemed to be working, then froze after 31 steps with just zsh: trace trap. If I change mps to cpu in the training command, then it works, but at roughly half the speed.

jkulhanek commented 1 year ago

Hi, would it be possible to run with the env variables TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 set and post the output?

jonahclarsen commented 1 year ago

Hi, would it be possible to run with the env variables TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 set and post the output?

Today I could get it to run for a few dozen steps until it crashed.

Here is the output: output.log

The exact command I ran: PYTORCH_ENABLE_MPS_FALLBACK=1 TORCH_LOGS="+dynamo" TORCHDYNAMO_VERBOSE=1 ns-train nerfacto --data data/nerfstudio/poster --machine.device-type mps --mixed-precision False --steps-per-save=20

For some reason the progress wasn't showing when I outputted it to a file (by calling with with &> output.log at the end), but I could see on the viewer that it was training for about 8 minutes, then it suddenly stopped (but it didn't log the progress or the crash). But on a separate run (that I didn't log to a file), here is the end of the output copied from the console before it crashed:


/Users/me/miniconda3/envs/nerfstudio/lib/python3.9/site-packages/nerfstudio/model_components/ray_samplers.py:345: UserWarning: The operator 'aten::searchsorted.Tensor' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/mps/MPSFallback.mm:13.)
  inds = torch.searchsorted(cdf, u, side="right")
[12:13:39] Printing max of 10 lines. Set flag --logging.local-writer.max-log-size=0 to disable line        writer.py:438
           wrapping.
Step (% Done)       Vis Rays / Sec       Train Iter (time)    ETA (time)           Train Rays / Sec
--------------------------------------------------------------------------------------------------------
100 (0.33%)         2 s, 624.522 ms      21 h, 47 m, 53 s     1.68 K
110 (0.37%)         2 s, 546.519 ms      21 h, 8 m, 35 s      1.74 K
120 (0.40%)         2 s, 729.247 ms      22 h, 39 m, 9 s      1.64 K
130 (0.43%)         2 s, 901.362 ms      1 d, 0 h, 4 m, 23 s  1.53 K
140 (0.47%)         2 s, 689.989 ms      22 h, 18 m, 43 s     1.65 K
150 (0.50%)         2 s, 517.066 ms      20 h, 52 m, 14 s     1.76 K
160 (0.53%)         2 s, 490.541 ms      20 h, 38 m, 37 s     1.79 K
170 (0.57%)         2 s, 515.878 ms      20 h, 50 m, 48 s     1.77 K
180 (0.60%)         2 s, 658.544 ms      22 h, 1 m, 17 s      1.69 K
181 (0.60%)         650.65               2 s, 744.640 ms      22 h, 44 m, 2 s      1.63 K
----------------------------------------------------------------------------------------------------
Viewer at: https://viewer.nerf.studio/versions/23-05-15-1/?websocket_url=ws://localhost:7007
zsh: trace trap  PYTORCH_ENABLE_MPS_FALLBACK=1 TORCH_LOGS="+dynamo" TORCHDYNAMO_VERBOSE=1```

jonahclarsen commented 1 year ago

@jkulhanek Oh and I forgot to mention, once it crashes, the nerfstudio viewer then says "Renderer Disconnected" and there is no more output to the console, but there remains a python3.9 process running that is using ~32% CPU and ~1.5GB that I have to Force Quit from Activity Monitor.

audiocommander commented 11 months ago

Hello,

just to chime in and bump this thread. I reached now exactly the same error and don't know how to continue as well. It feels like we're 99% there. But I haven't heard or read from anyone who actually managed to get it running on an M1/M2.

Now I can get it to install, and using the training command from #2436 I got past a Torch not compiled with CUDA enabled error. That command is: PYTORCH_ENABLE_MPS_FALLBACK=1 ns-train nerfacto --data data/nerfstudio/poster --machine.device-type mps --mixed-precision False But then I got this error: User specified an unsupported autocast device_type 'mps'

I successfully installed PyTorch with Metal support and TensorFlow with Metal Plugin (MPS). Testing scripts work.

$ conda list | grep tensorflow tensorflow-aarch64 1.2 pypi_0 pypi

Cheers! Michael

jonahclarsen commented 11 months ago

@audiocommander I finally got it working by following the instructions on this page to install from source: https://docs.nerf.studio/quickstart/installation.html (skipping cuda, tinycudann, and installing the non "+cu118" versions of pytorch/torchvision).

So:

conda create --name nerfstudio -y python=3.8
conda activate nerfstudio
python -m pip install --upgrade pip
pip uninstall torch torchvision functorch tinycudann
pip install torch==2.0.1 torchvision==0.15.2
git clone https://github.com/nerfstudio-project/nerfstudio.git
cd nerfstudio
pip install --upgrade pip setuptools
pip install -e .

richardtallent commented 11 months ago

Yeah, that didn't work here on an M3 running Sonoma. :(

First, instructions above assume one has "conda" installed. I don't. I installed Miniconda.

Some dependencies needed Python 3.9, so the "3.8" did not work. I ended up using 3.11 (which I already have installed) instead.

Then pip install -e bailed because Open3D was MIA.

Then I went down a rabbit hole based on this thread of trying to compile Open3D locally. That then led to variations of this error.

Finally gave up.

Somewhere along the way (can't remember when) I had to brew install a FORTRAN(?!!?) compiler. Sheesh. Haven't seen that language in 20-30 years.

jkulhanek commented 11 months ago

I think we can make open3d a soft dependency. It is only needed for export.

Isabel-S commented 9 months ago

@jonahclarsen this worked for me! w/ a M3.

Itox001 commented 4 months ago

@jonahclarsen the installation worked for me as well, I used the versions from https://github.com/nerfstudio-project/nerfstudio/blob/main/docs/quickstart/installation.md#dependencies for a more recent setup.

I had to install Open3D which was not found when using python 3.12, but on 3.11 was installed no problems. Also, had to downgrade Numpy to <2.0.0. I think that was it.

I could now train the model without errors, but unfortunately the output seems to not work. The viewer shows the camera angles, but there is no model rescontruction, just a blurry background after 2000 iterations. Doing more is painful because the support for MPS is still very lacking, but the same number of iterations provides an acceptable model using OpenSplat, so I am not sure what is the problem.

Noezor commented 6 days ago

@jonahclarsen 's solution didn't work for me on M1. I got (again) the numpy 2 error so changed it to

conda create --name nerfstudio -y python=3.8
conda activate nerfstudio
python -m pip install --upgrade pip
pip uninstall torch torchvision functorch tinycudann numpy
pip install numpy==1.16
pip install torch==2.0.1 torchvision==0.15.2
git clone https://github.com/nerfstudio-project/nerfstudio.git
cd nerfstudio
pip install --upgrade pip setuptools
pip install -e .

but I am now getting hit with the cuda error:

Torch not compiled with CUDA enabled

Fairly annoying :( will try to spend more time on it

Noezor commented 6 days ago

So, it turns out that the install actually works. It's just that the commands (in particular ns-train in the tutorial) start by default on cuda devices. If you run the ns-train with the command

ns-train <whichever parameters> --machine.device_type cpu

it runs (but takes ages on my M1 for the poster)

I'll do MRs to improve the documentation and the default parameters.

nerfstudio-project / nerfstudio

Mac Installation Instructions? #2438