kijai / ComfyUI-SUPIR

SUPIR upscaling wrapper for ComfyUI
Other
1.54k stars 83 forks source link

No longer working on MPS - 'aten::upsample_bicubic2d.out' is not currently implemented for the MPS device. #47

Open discordinated opened 7 months ago

discordinated commented 7 months ago

I get this error on every run now, some commit in the last few days or so must have changed something. I tried changing the bicubic on line 213 of nodes.py to bilinear, that runs - however output is then just pure noise.

Any advice?

Error occurred when executing SUPIR_Upscale:

The operator 'aten::upsample_bicubic2d.out' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

File "/Users/s/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes.py", line 213, in process
resized_image = F.interpolate(image, size=(new_height, new_width), mode='bicubic', align_corners=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/miniconda3/lib/python3.11/site-packages/torch/nn/functional.py", line 4035, in interpolate
return torch._C._nn.upsample_bicubic2d(input, output_size, align_corners, scale_factors)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

kijai commented 7 months ago

Hmm we can just do the image resize on cpu then, it's possible I changed the order earlier, if it worked before, I'll change moving to device after the resize now.

discordinated commented 7 months ago

It appears to start to work with export PYTORCH_ENABLE_MPS_FALLBACK=1 ran before launching comfyui

However it then gives this error after the executing encode task queues

Error occurred when executing SUPIR_Upscale:

The size of tensor a (32) must match the size of tensor b (128) at non-singleton dimension 3

File "/Users/s/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes.py", line 233, in process
samples = self.model.batchify_sample(imgs, caps, num_steps=steps,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/miniconda3/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/custom_nodes/ComfyUI-SUPIR/SUPIR/models/SUPIR_model.py", line 143, in batchify_sample
_samples = self.sampler(denoiser, noised_z, cond=c, uc=uc, x_center=z_stage1, control_scale=control_scale,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/s/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 511, in __call__
x_next[:, :, hi:hi_end, wi:wi_end] += _x * tile_weights
~~~^~~~~~~~~~~~~~

If I then disable tiled sampling it then completes it's run but the output is pure noise.

kijai commented 7 months ago

Did you have tiled sampling working on MPS before? I know someone else who tried and it always failed for them.

discordinated commented 7 months ago

I'm not 100% sure, I think it was probably off. I don't have anything saved to check however.

kijai commented 7 months ago

I'm not 100% sure, I think it was probably off. I don't have anything saved to check however.

Ok, well I moved the image resize to before image is moved to device (which in your case is MPS), maybe that helps with the initial error at least.

discordinated commented 7 months ago

Thanks will test and report back

awfulcode commented 3 months ago

For what it's worth, I fixed it by changing the upsample method to bilinear. On nodes_v2.py, change line 239 to decoded_out = F.interpolate(decoded_out, size=(orig_H, orig_W), mode="bilinear")

Tolkinz commented 1 month ago

For what it's worth, I fixed it by changing the upsample method to bilinear. On nodes_v2.py, change line 239 to decoded_out = F.interpolate(decoded_out, size=(orig_H, orig_W), mode="bilinear")

Where can I find nodes_v2.py file? I am having this problem. Fallback option resulted in excessive memory pressure, almost not usable. Thanks.