Add ZoeDetector

[pdoane]

Note that this contains what appears to be a full midas repository which could be pruned (e.g. there are Android/iOS example apps). This is the minimal change to the ControlNet 1.1 implementation to match the style of the rest of the package. There is also a significant amount of debug information printed to the console that would be nice to disable.

The depth image is noticeable better than Midas so I think this worth including.

[patrickvonplaten]

Thanks a lot for adding this! It would be great if we could prune the PR a bit:

• Don't think any of: .zoedepth.data .zoedepth.trainer folders are needed.

Also it'd be amazing if we could clean up the midas repo a lot. Couldn't we just use this short part of the midas repo that SD 2 uses as well: https://github.com/Stability-AI/stablediffusion/tree/main/ldm/modules/midas or do we need more code ?

[pdoane]

The midas_repo is now similarly pruned compared to SD2. This is a different version which adds the backbone support. I did not collapse the directory structure and left the project README/LICENSE.

I also disabled the print calls, but I think there is a bug in the implementation. This line seems suspect:

https://github.com/pdoane/controlnet_aux/blob/pdoane/zoe-detector/src/controlnet_aux/zoe/zoedepth/models/base_models/midas.py#L180

The debugging lines show that width/height are being swapped.

[pdoane]

Issue opened upstream about the possible width/height swap - https://github.com/lllyasviel/ControlNet-v1-1-nightly/issues/29

[patrickvonplaten]

Final comment here would be to remove the matplotlib dependency (we don't have one and don't need one IMO). Also think in some files we should try to import locally no? See: https://github.com/patrickvonplaten/controlnet_aux/pull/26#discussion_r1170966332

[pdoane]

Made both changes. Would you like me to squash/rebase before merging?

[patrickvonplaten]

Looks good!

[pdoane]

@patrickvonplaten - Do you have an ETA for an updated release?