kijai / ComfyUI-SUPIR

SUPIR upscaling wrapper for ComfyUI
Other
1.2k stars 66 forks source link

Unwanted resize after SUPIR first Stage (Denoiser) #94

Open Poukpalaova opened 3 months ago

Poukpalaova commented 3 months ago

I got a strange behavior after the SUPIR First Stage (Denoiser) and want to know if it's normal. I didn't do much test but I can say that most of the them, the image getting out of this first stage is deformed. Not by much, only by little. One of the side, not always the same, will change. It probably not doing it on perfectly divisible by 64 number tho. My work around for now is to do a get resolution on the input picture, and a resize of both width and height of the output image to that input size value and it work great for now.

kijai commented 3 months ago

SUPIR requires the image to be divisible by 32, which I'm doing automatically in the code. The original size is saved in the latent and carried all the way to the end, where the final decode resizes it back. However the output image from the First Stage -node is not resized back, as in normal use it would then have to again be fixed in the encode node.

Is there a use case for the first stage image output? I myself see it mostly just as a preview. To avoid all this altogether you can also make sure the initial input is divisible by 32.

Poukpalaova commented 3 months ago

SUPIR requires the image to be divisible by 32, which I'm doing automatically in the code. The original size is saved in the latent and carried all the way to the end, where the final decode resizes it back. However the output image from the First Stage -node is not resized back, as in normal use it would then have to again be fixed in the encode node.

Is there a use case for the first stage image output? I myself see it mostly just as a preview. To avoid all this altogether you can also make sure the initial input is divisible by 32.

You saying that the final decode resizes it back, but I still see a small difference. I noticed because I did a compare workflow and we can see some pixel missing sometime on top/bottom or left/right. Actually they are not missing, the picture is stretched to reach that step of 32 but I don't see it getting back to original at the end. And I noticed that this is done in the first stage denoise (nice work there actually for the denoiser). So in conclusion it is not a very big deal as I a resise after the total supir process to the dim of the input image and it work great.

kijai commented 3 months ago

SUPIR requires the image to be divisible by 32, which I'm doing automatically in the code. The original size is saved in the latent and carried all the way to the end, where the final decode resizes it back. However the output image from the First Stage -node is not resized back, as in normal use it would then have to again be fixed in the encode node. Is there a use case for the first stage image output? I myself see it mostly just as a preview. To avoid all this altogether you can also make sure the initial input is divisible by 32.

You saying that the final decode resizes it back, but I still see a small difference. I noticed because I did a compare workflow and we can see some pixel missing sometime on top/bottom or left/right. Actually they are not missing, the picture is stretched to reach that step of 32 but I don't see it getting back to original at the end. And I noticed that this is done in the first stage denoise (nice work there actually for the denoiser). So in conclusion it is not a very big deal as I a resise after the total supir process to the dim of the input image and it work great.

With what dimensions does this happen? It's working for me with this setup:

image In the case it autoresizes, the decoder at the end should say:

image

DanielXu123 commented 1 month ago

@kijai Firstly, Thanks for your work . I tried to change the sampler from RestoreSampler to TiledSampler, the former one won't return any error, but the Tiled Sampler will return error as following. I think it's related to the image size I guess, could you give any solutions or advice?


Error occurred when executing SUPIR_sample:

The size of tensor a (48) must match the size of tensor b (128) at non-singleton dimension 3

File "/data/AIGC/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/data/AIGC/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/data/AIGC/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/data/AIGC/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 494, in sample
_samples = self.sampler(denoiser, noised_z, cond=positive[i], uc=negative[i], x_center=sample.unsqueeze(0), control_scale=control_scale_end,
File "/data/AIGC/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 743, in __call__
x_next[:, :, hi:hi_end, wi:wi_end] += _x * tile_weights