search

HengyiWang / spann3r

[3DV'25] 3D Reconstruction with Spatial Memory
https://hengyiwang.github.io/projects/spanner
737 stars 34 forks source link

About the demo error #12

Open ZhenyuSun-Walker opened 1 month ago

ZhenyuSun-Walker commented 1 month ago

Hello Sir, thank you for your brilliant work. I wonder how to deal with the issue when I run the demo.py, and get the error information below

Traceback (most recent call last):
  File "/home/sunzhenyu/Projects/spann3r/demo.py", line 157, in <module>
    main(args)
  File "/home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/sunzhenyu/Projects/spann3r/demo.py", line 50, in main
    batch = dataloader.__iter__().__next__()
  File "/home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 701, in __next__
    data = self._next_data()
  File "/home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 757, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 52, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/sunzhenyu/Projects/spann3r/dust3r/datasets/base/base_stereo_view_dataset.py", line 80, in __getitem__
    views = self._get_views(idx, resolution, self._rng)
  File "/home/sunzhenyu/Projects/spann3r/spann3r/datasets/demo.py", line 86, in _get_views
    rgb_image, depthmap, intrinsics = self._crop_resize_if_necessary(
  File "/home/sunzhenyu/Projects/spann3r/dust3r/datasets/base/base_stereo_view_dataset.py", line 187, in _crop_resize_if_necessary
    image, depthmap, intrinsics = cropping.rescale_image_depthmap(image, depthmap, intrinsics, target_resolution)
  File "/home/sunzhenyu/Projects/spann3r/dust3r/datasets/utils/cropping.py", line 70, in rescale_image_depthmap
    image = image.resize(output_resolution, resample=lanczos)
  File "/home/sunzhenyu/Projects/spann3r/dust3r/datasets/utils/cropping.py", line 45, in resize
    return ImageList(self._dispatch('resize', *args, **kwargs))
  File "/home/sunzhenyu/Projects/spann3r/dust3r/datasets/utils/cropping.py", line 51, in _dispatch
    return [getattr(im, func)(*args, **kwargs) for im in self.images]
  File "/home/sunzhenyu/Projects/spann3r/dust3r/datasets/utils/cropping.py", line 51, in <listcomp>
    return [getattr(im, func)(*args, **kwargs) for im in self.images]
  File "/home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages/PIL/Image.py", line 2297, in resize
    if self.size == size and box == (0, 0) + self.size:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
HengyiWang commented 1 month ago

Hi @ZhenyuSun-Walker, can you check your pillow version?

https://github.com/HengyiWang/spann3r/blob/5f2c293130fc1476d80a7deab842bada7eecc096/requirements.txt#L14

ZhenyuSun-Walker commented 1 month ago

Sir, actually my pillow version satisfies the requirement. image

ZhenyuSun-Walker commented 1 month ago

And actually I cannot install the required version of open3d, thus I install the open3d=0.18.0 instead and it has done succesfully. However, I doubt that the error could be caused by the open3d installation.

(spann3r) sunzhenyu@mercury:~/Projects/spann3r$ pip install -U -f https://www.open3d.org/docs/latest/getting_started.html open3d
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Looking in links: https://www.open3d.org/docs/latest/getting_started.html
Requirement already satisfied: open3d in /home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages (0.18.0)
Collecting open3d
  Downloading https://github.com/isl-org/Open3D/releases/download/main-devel/open3d-0.18.0%2Bf4e1fa9-app-windows-amd64.zip (50.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.4/50.4 MB 31.4 kB/s eta 0:00:00
ERROR: open3d from https://github.com/isl-org/Open3D/releases/download/main-devel/open3d-0.18.0%2Bf4e1fa9-app-windows-amd64.zip does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.
ZhenyuSun-Walker commented 1 month ago

Hello sir, Would you mind helping me solve the problems above?

HengyiWang commented 1 month ago

Hi @ZhenyuSun-Walker, this error should be related to the issue here: https://github.com/python-pillow/Pillow/issues/8220. I am not sure why you have this issue with pillow 10.3.0 but I have updated the code to fix it temporally: https://github.com/HengyiWang/spann3r/blob/a454b28ffe4840e37de98897739d98513c99b789/dust3r/datasets/utils/cropping.py#L70 Can you pull and see if this resolve your issue?

ZhenyuSun-Walker commented 1 month ago

Thank you for your timely fix! Now this issue is solved successfully, but now after I run the demo.py, the result is like: image

HengyiWang commented 1 month ago

Can you check which line causes this segmentation fault? Is it related to open3d?

ZhenyuSun-Walker commented 1 month ago

The whole messgae is shown as below

(spann3r) sunzhenyu@mercury:~/Projects/spann3r$ python demo.py --demo_path ./examples/s00567 --kf_every 10 --vis
... loading model from ./checkpoints/DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth
/home/sunzhenyu/Projects/spann3r/dust3r/model.py:35: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  ckpt = torch.load(model_path_or_url, map_location='cpu')
instantiating : AsymmetricCroCo3DStereo(enc_depth=24, dec_depth=12, enc_embed_dim=1024, dec_embed_dim=768, enc_num_heads=16, dec_num_heads=12, pos_embed='RoPE100', patch_embed_cls='PatchEmbedDust3R', img_size=(512, 512), head_type='dpt', output_mode='pts3d', depth_mode=('exp', -inf, inf), conf_mode=('exp', 1, inf), landscape_only=True)
<All keys matched successfully>
/home/sunzhenyu/Projects/spann3r/demo.py:43: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  model.load_state_dict(torch.load(args.ckpt_path)['model'])
Loading image: ./examples/s00567/frame_00000.jpg
Loading image: ./examples/s00567/frame_00010.jpg
Loading image: ./examples/s00567/frame_00020.jpg
Loading image: ./examples/s00567/frame_00030.jpg
Loading image: ./examples/s00567/frame_00040.jpg
Loading image: ./examples/s00567/frame_00050.jpg
Loading image: ./examples/s00567/frame_00060.jpg
Loading image: ./examples/s00567/frame_00070.jpg
Loading image: ./examples/s00567/frame_00080.jpg
Loading image: ./examples/s00567/frame_00090.jpg
Loading image: ./examples/s00567/frame_00100.jpg
Loading image: ./examples/s00567/frame_00110.jpg
Loading image: ./examples/s00567/frame_00120.jpg
Loading image: ./examples/s00567/frame_00130.jpg
Loading image: ./examples/s00567/frame_00140.jpg
Loading image: ./examples/s00567/frame_00150.jpg
Loading image: ./examples/s00567/frame_00160.jpg
Loading image: ./examples/s00567/frame_00170.jpg
Loading image: ./examples/s00567/frame_00180.jpg
Loading image: ./examples/s00567/frame_00190.jpg
Loading image: ./examples/s00567/frame_00200.jpg
Loading image: ./examples/s00567/frame_00210.jpg
Loading image: ./examples/s00567/frame_00220.jpg
Loading image: ./examples/s00567/frame_00230.jpg
Loading image: ./examples/s00567/frame_00240.jpg
Loading image: ./examples/s00567/frame_00250.jpg
Loading image: ./examples/s00567/frame_00260.jpg
Loading image: ./examples/s00567/frame_00270.jpg
Loading image: ./examples/s00567/frame_00280.jpg
Loading image: ./examples/s00567/frame_00290.jpg
Loading image: ./examples/s00567/frame_00300.jpg
Loading image: ./examples/s00567/frame_00310.jpg
Loading image: ./examples/s00567/frame_00320.jpg
Loading image: ./examples/s00567/frame_00330.jpg
Loading image: ./examples/s00567/frame_00340.jpg
Loading image: ./examples/s00567/frame_00350.jpg
Loading image: ./examples/s00567/frame_00360.jpg
Loading image: ./examples/s00567/frame_00370.jpg
Loading image: ./examples/s00567/frame_00380.jpg
Loading image: ./examples/s00567/frame_00390.jpg
Loading image: ./examples/s00567/frame_00400.jpg
Loading image: ./examples/s00567/frame_00410.jpg
Loading image: ./examples/s00567/frame_00420.jpg
Loading image: ./examples/s00567/frame_00430.jpg
Loading image: ./examples/s00567/frame_00440.jpg
Loading image: ./examples/s00567/frame_00450.jpg
Loading image: ./examples/s00567/frame_00460.jpg
Loading image: ./examples/s00567/frame_00470.jpg
Loading image: ./examples/s00567/frame_00480.jpg
Loading image: ./examples/s00567/frame_00490.jpg
Loading image: ./examples/s00567/frame_00500.jpg
Loading image: ./examples/s00567/frame_00510.jpg
Loading image: ./examples/s00567/frame_00520.jpg
Loading image: ./examples/s00567/frame_00530.jpg
Loading image: ./examples/s00567/frame_00540.jpg
Loading image: ./examples/s00567/frame_00550.jpg
Loading image: ./examples/s00567/frame_00560.jpg
Loading image: ./examples/s00567/frame_00570.jpg
Started reconstruction for s00567
/home/sunzhenyu/Projects/spann3r/spann3r/model.py:503: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
/home/sunzhenyu/Projects/spann3r/spann3r/model.py:320: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  with torch.cuda.amp.autocast(enabled=False):
Similarity detected: tensor(0.9644, device='cuda:0')
Similarity detected: tensor(0.9938, device='cuda:0')
Similarity detected: tensor(0.9816, device='cuda:0')
Similarity detected: tensor(0.9570, device='cuda:0')
Similarity detected: tensor(0.9646, device='cuda:0')
Similarity detected: tensor(0.9776, device='cuda:0')
Similarity detected: tensor(0.9575, device='cuda:0')
Similarity detected: tensor(0.9826, device='cuda:0')
Similarity detected: tensor(0.9533, device='cuda:0')
Similarity detected: tensor(0.9566, device='cuda:0')
Similarity detected: tensor(0.9539, device='cuda:0')
Similarity detected: tensor(0.9766, device='cuda:0')
Similarity detected: tensor(0.9662, device='cuda:0')
Similarity detected: tensor(0.9568, device='cuda:0')
Similarity detected: tensor(0.9541, device='cuda:0')
Similarity detected: tensor(0.9564, device='cuda:0')
Similarity detected: tensor(0.9559, device='cuda:0')
Similarity detected: tensor(0.9798, device='cuda:0')
Similarity detected: tensor(0.9606, device='cuda:0')
Similarity detected: tensor(0.9750, device='cuda:0')
Similarity detected: tensor(0.9625, device='cuda:0')
Similarity detected: tensor(0.9804, device='cuda:0')
Similarity detected: tensor(0.9664, device='cuda:0')
Similarity detected: tensor(0.9659, device='cuda:0')
Similarity detected: tensor(0.9512, device='cuda:0')
Similarity detected: tensor(0.9575, device='cuda:0')
Similarity detected: tensor(0.9837, device='cuda:0')
Similarity detected: tensor(0.9823, device='cuda:0')
Similarity detected: tensor(0.9764, device='cuda:0')
Similarity detected: tensor(0.9728, device='cuda:0')
Similarity detected: tensor(0.9729, device='cuda:0')
Similarity detected: tensor(0.9610, device='cuda:0')
Similarity detected: tensor(0.9532, device='cuda:0')
Similarity detected: tensor(0.9755, device='cuda:0')
Similarity detected: tensor(0.9578, device='cuda:0')
Finished reconstruction for s00567, FPS: 24.45
Segmentation fault (core dumped)
HengyiWang commented 1 month ago

I mean can you use either debugging mode or print statement to see which line causes this error?

umutcanustuntas commented 1 month ago

Same issue here. I believe it is an open3d related issue, a quick search shows too many similar issues (such as this numpy issue In my case the error occurs in the following line: https://github.com/HengyiWang/spann3r/blob/a454b28ffe4840e37de98897739d98513c99b789/demo.py#L141 Downgrading numpy to 1.26.4 seems to solve the issue.

HengyiWang commented 1 month ago

Thanks @umutcanustuntas, that is quite helpful! I have checked my local environment, and yes, I use numpy=1.26.4 - will update the requirements.txt shortly.

ZhenyuSun-Walker commented 1 month ago

Thank you @umutcanustuntas @HengyiWang ！

ZhenyuSun-Walker commented 1 month ago

However， I found a new erroe

[Open3D WARNING] GLFW Error: X11: The DISPLAY environment variable is missing
[Open3D WARNING] Failed to initialize GLFW
Traceback (most recent call last):
  File "/home/sunzhenyu/Projects/spann3r/demo.py", line 157, in <module>
    main(args)
  File "/home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/sunzhenyu/Projects/spann3r/demo.py", line 147, in main
    camera_parameters = find_render_cam(pcd)
  File "/home/sunzhenyu/Projects/spann3r/spann3r/tools/vis.py", line 97, in find_render_cam
    opt.point_size = 1
AttributeError: 'NoneType' object has no attribute 'point_size'
Yuxin-Du-Lab commented 3 weeks ago

However， I found a new erroe

[Open3D WARNING] GLFW Error: X11: The DISPLAY environment variable is missing
[Open3D WARNING] Failed to initialize GLFW
Traceback (most recent call last):
  File "/home/sunzhenyu/Projects/spann3r/demo.py", line 157, in <module>
    main(args)
  File "/home/sunzhenyu/anaconda3/envs/spann3r/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/sunzhenyu/Projects/spann3r/demo.py", line 147, in main
    camera_parameters = find_render_cam(pcd)
  File "/home/sunzhenyu/Projects/spann3r/spann3r/tools/vis.py", line 97, in find_render_cam
    opt.point_size = 1
AttributeError: 'NoneType' object has no attribute 'point_size'

Same error report