isl-org / MiDaS

Code for robust monocular depth estimation described in "Ranftl et. al., Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer, TPAMI 2022"
MIT License
4.25k stars 597 forks source link

Tuple error, as of today #246

Open WorldofDepth opened 8 months ago

WorldofDepth commented 8 months ago

Hi, this ran for me fine yesterday, but today gives the following error. Any ideas on how to solve it? I am using timm 0.6.13. Thank you for any help!

Initialize
Device: cuda
/usr/local/lib/python3.10/dist-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3526.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Model loaded, number of parameters = 345M
Start processing
  Processing input/13a-s.png (1/1)
    Input resized to 512x608 before entering the encoder
Traceback (most recent call last):
  File "/content/MiDaS/run.py", line 276, in <module>
    run(args.input_path, args.output_path, args.model_weights, args.model_type, args.optimize, args.side, args.height,
  File "/content/MiDaS/run.py", line 154, in run
    prediction = process(device, model, model_type, image, (net_w, net_h), original_image_rgb.shape[1::-1],
  File "/content/MiDaS/run.py", line 61, in process
    prediction = model.forward(sample)
  File "/content/MiDaS/midas/dpt_depth.py", line 166, in forward
    return super().forward(x).squeeze(dim=1)
  File "/content/MiDaS/midas/dpt_depth.py", line 114, in forward
    layers = self.forward_transformer(self.pretrained, x)
  File "/content/MiDaS/midas/backbones/beit.py", line 15, in forward_beit
    return forward_adapted_unflatten(pretrained, x, "forward_features")
  File "/content/MiDaS/midas/backbones/utils.py", line 86, in forward_adapted_unflatten
    exec(f"glob = pretrained.model.{function_name}(x)")
  File "<string>", line 1, in <module>
  File "/content/MiDaS/midas/backbones/beit.py", line 125, in beit_forward_features
    x = blk(x, resolution, shared_rel_pos_bias=rel_pos_bias)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/content/MiDaS/midas/backbones/beit.py", line 102, in block_forward
    x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x), resolution,
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/content/MiDaS/midas/backbones/beit.py", line 81, in attention_forward
    attn = attn + self._get_rel_pos_bias(window_size)
  File "/content/MiDaS/midas/backbones/beit.py", line 47, in _get_rel_pos_bias
    new_sub_table = F.interpolate(old_sub_table, size=(new_height, new_width), mode="bilinear")
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py", line 3924, in interpolate
    raise TypeError(
TypeError: expected size to be one of int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int], but got size with types [<class 'numpy.int64'>, <class 'numpy.int64'>]
WorldofDepth commented 8 months ago

Update: apparently torch 2.1.0 is the problem; reverting to torch 2.0.1 using !pip install torch==2.0.1 fixes it.

On a related note, it was already clear that MiDaS requires an older version of timm to function (!pip install timm==0.6.13).

This is for using MiDaS inside a Google Colab python notebook.

Developers, if you are able, please update for torch and timm compatibility!

Arslan-Mehmood1 commented 6 months ago

Hy, I just encountered the same error, but the above solution didn't work for me.

Installing the following packages after git clone, worked for me in performing the inference. !pip install torchvision==0.14.0 !pip install timm==0.6.12 !pip install einops==0.6.0

Colab Notebook Link: https://colab.research.google.com/drive/1wHy3aNNdYR44TqmAJ9rYT_98XbtlB_j-?usp=sharing