search

invoke-ai / InvokeAI

Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
23.64k stars 2.43k forks source link

[bug][regression?]: Depth (Zoe) processor fails with TypeError #5083

Open keturn opened 12 months ago

keturn commented 12 months ago

Is there an existing issue for this?

OS

Linux

GPU

cuda

VRAM

12GB

What version did you experience this issue on?

1c7ea57492802eaf19f81ada9ed545a2e618add8

What happened?

Traceback:

  File "site-packages/controlnet_aux/zoe/zoedepth/models/base_models/midas.py", line 263, in forward
    x = self.prep(x)
       ^^^^^^^^^^^^   
  File "site-packages/controlnet_aux/zoe/zoedepth/models/base_models/midas.py", line 187, in __call__
    return self.normalization(self.resizer(x))
                              ^^^^^^^^^^^^^^^
  File "site-packages/controlnet_aux/zoe/zoedepth/models/base_models/midas.py", line 174, in __call__
    return nn.functional.interpolate(x, (height, width), mode='bilinear', align_corners=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/torch/nn/functional.py", line 3924, in interpolate
    raise TypeError(                                                                                                                                                        
TypeError: expected size to be one of int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int], but got size with types [<class 'numpy.int64'>, <class 'numpy.int64'>]

though this traceback says it is in resizer, it does not change when I set Resize Mode to Crop or Fill instead of Resize.

The Midas depth processor works without error.

keturn commented 12 months ago

This must be a regression, since I'm sure we had a working zoe depth processor at some point.

Millu commented 12 months ago

Other UIs are reporting the same error:

The A111 issue references torch 2.1 as a possible cause 😬

psychedelicious commented 12 months ago

Yeah, it's a torch 2.1 issue, introduced in this PR: https://github.com/pytorch/pytorch/pull/99243#issuecomment-1646736129

I was about to create a PR to fix it, but I see it was actually already done: https://github.com/pytorch/pytorch/pull/110778

So I think we will just need to wait for the next release of torch.

psychedelicious commented 12 months ago

A quicker fix might be to update controlnet-aux to use plain integers. It's a good handful of spots where this would be needed: https://github.com/search?q=repo%3Apatrickvonplaten%2Fcontrolnet_aux+interpolate%28&type=code

psychedelicious commented 11 months ago

torch 2.1.1 was released yesterday - it does not include the fix. Here's the 2.1.1 tag with that code: https://github.com/pytorch/pytorch/blob/v2.1.1/torch/nn/functional.py#L3777-L3783

Vargol commented 11 months ago

Look like it going to be fixed in 2.2, the merge has been pulled to the 2.2 release branch..

https://github.com/pytorch/pytorch/blob/e57f089704ae9fa0f57f18c6122ce645457b183a/torch/nn/functional.py#L3797

psychedelicious commented 9 months ago

This is fixed in the just-released torch 2.2.