search

GuHuangAI / DiffusionEdge

Code for AAAI 2024 paper: "DiffusionEdge: Diffusion Probabilistic Model for Crisp Edge Detection"
Apache License 2.0
190 stars 18 forks source link

How to infer for images of small size #4

Open GiannakopoulosIlias opened 6 months ago

GiannakopoulosIlias commented 6 months ago

Hello,

I have a custom datasets of jpgs of a small size (38x47 to be precise).

Is it possible to adjust the network to infer the edges for these images?

If I run without any modifications in the network, I get this error:

File "./DiffusionEdge-main/denoising_diffusion_pytorch/mask_cond_unet.py", line 952, in forward x = torch.cat((x, h.pop()), dim = 1) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 4 but got size 5 for tensor number 1 in the list.

Thanks!

GuHuangAI commented 6 months ago

@GiannakopoulosIlias Hello, I have two suggestions: (1) Since the model is trained with the image size of 320x320, it can not deal with images that have smaller sizes. Without retraining the model, you can try to resize the smaller images to larger sizes. I guess the performance may be not good. (2) The better way to obtain correct edges is to retrain the diffusionedge with a crop size of 32x32. Besides, I'm a little curious what your images look like.

GiannakopoulosIlias commented 6 months ago

Hello, thanks a lot for the inputs on this.

For (1) I agree that an interpolation might lead it sub-optimal results. For (2) we have 2D quantitative maps and we can export them to jpgs. Here is how the python plot looks like.
Do you think retraining the network with one of the available datasets would work well for these type of edges? Unfortunately we don't have any ground-truth edge mask.

2024-01-09_14-30

GuHuangAI commented 6 months ago

I think diffusionedge can handle this type of image when training on the similar datasets (small images), however, I don't know if there is such a dataset. If you want to train diffusionedge on this type of dataset (32x32), you do not have to use latent diffusion model, but just train the model in the image space.

GiannakopoulosIlias commented 6 months ago

I think diffusionedge can handle this type of image when training on the similar datasets (small images), however, I don't know if there is such a dataset. If you want to training diffusion edge on this type of dataset (32x32), you do not have to use latent diffusion model, but just train the model in the image space.

Thanks again! I will experiment a bit further with resizing as it seems simpler for now.