Running on Apple Mac M2

[nmonet]

Good jobs guys ! Very impressive results.

I confirm that it's working on Apple Mac ( with Apple Silicon ), I try with more than 8 images with no error. 3 images, 6 images pairs, run in 6 sec + 15 sec 8 images, 56 images pairs, run in 70 sec + 90 sec

PYTORCH_ENABLE_MPS_FALLBACK=1 python3 demo.py --weights checkpoints/DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth --device mps

Did you plan to release some samples with a more bigger number of images ?

[liujun3660105]

how do you install it. I have some error when conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia

[nmonet]

conda install pytorch::pytorch torchvision torchaudio -c pytorch for Pytorch 2.2 on MacOS.

[vincent-leroy]

Thanks ! We are working on something else currently, but yeah the default scene graph is complete (all images are connected to all other images) so all pairs will be computed, and used for the global alignment step. Complexity grows quadratically w.r.t number of images, but you can have a look at https://github.com/naver/dust3r/issues/1 that explains how to change the scene graph to reduce the number of edges (pairs to be considered). These are only sliding window and one-ref strategies, but, whenever you have a large number of images, you probably want to use retrieval to filter out bad pairs, build a sparse scene graph and only optimize for it (e.g. only considering compatible views). Note that DUSt3R itself can be used to score pairs based on the confidence of the predictions (unrelated image pairs usually have a globally low confidence) we had something for that but it did not make it into the release. We might add it at some point.

[HARSHA839]

Hi,

Were you able to generate the novel views on Mac?

[yeraypabon]

I can confirm it works on Mac M3 too. Is there any way the code uses the parallel framework mac provides or it's only cuda?