lower GPU VRAM usage , upscalers integration

[mr-lab]

i think using timm==0.9.2 will lower VRAM usage

trying to implement this project to sdwebui in zoedepth\models\base_models\midas.py i changed the line 341: midas = torch.hub.load("intel-isl/MiDaS", midas_model_type, pretrained=use_pretrained_midas, force_reload=force_reload)

to : midas = torch.hub.load("AyaanShah2204/MiDaS", midas_model_type, pretrained=use_pretrained_midas, force_reload=force_reload)

it's using about 6-7gb on my end plus i'm using the preinstalled dependencies of sdwebui 1.6

that could be the improvement .

edit : thinking about it ,the input image gets upscaled using resolutions only , why not use AI upscalers like 4x-UltraSharp .. etc i will work on doing that soon and see the results

[zhyever]

Thanks for your effort. I would have a try about this modification. It's really helpful.

The image upsample idea is amazing for me. Would it be possible for us that even if given a blur image, we could still give a high-resolution image?

[mr-lab]

i did test 4x-ultrasharp after i generated an image with SDWEBUI the image was 768*1344 upscaled 2x the results appears to be very good for depth generation , the image was clear not blurry for blur image that i think you will have to use img2img to clear it , here is an example just posted using COntrolnet SOftedge HED + controlnet TIle no preprocesor. https://www.reddit.com/r/StableDiffusion/comments/18msbgb/magnific_ai_but_it_is_free_a1111/ i think you can just upscale it and send it as an image no need to replace image_hr = F.interpolate(image, image_resolution, mode='bicubic', align_corners=True) it's the same results if you use it or not it's good to have so that the patches will always cover the image