python-pillow / Pillow

Python Imaging Library (Fork)
https://python-pillow.org
Other
12.28k stars 2.23k forks source link

How to approximate pytorch's gaussian blur using PIL? #6744

Closed vedantroy closed 1 year ago

vedantroy commented 1 year ago

Pytorch has the method:

    dtype = image.dtype if torch.is_floating_point(image) else torch.float32
    kernel = _get_gaussian_kernel2d(kernel_size, sigma, dtype=dtype, device=image.device)
    kernel = kernel.expand(image.shape[-3], 1, kernel.shape[0], kernel.shape[1])

    image, need_cast, need_squeeze, out_dtype = _FT._cast_squeeze_in(image, [kernel.dtype])

    # padding = (left, right, top, bottom)
    padding = [kernel_size[0] // 2, kernel_size[0] // 2, kernel_size[1] // 2, kernel_size[1] // 2]
    output = torch_pad(image, padding, mode="reflect")
    output = conv2d(output, kernel, groups=output.shape[-3])

which does a gaussian blur of a tensor using the following helper methods:

def _get_gaussian_kernel1d(kernel_size: int, sigma: float, dtype: torch.dtype, device: torch.device) -> torch.Tensor:
    lim = (kernel_size - 1) / (2 * math.sqrt(2) * sigma)
    x = torch.linspace(-lim, lim, steps=kernel_size, dtype=dtype, device=device)
    kernel1d = torch.softmax(-x.pow_(2), dim=0)
    return kernel1d

def _get_gaussian_kernel2d(
    kernel_size: List[int], sigma: List[float], dtype: torch.dtype, device: torch.device
) -> torch.Tensor:
    kernel1d_x = _get_gaussian_kernel1d(kernel_size[0], sigma[0], dtype, device)
    kernel1d_y = _get_gaussian_kernel1d(kernel_size[1], sigma[1], dtype, device)
    kernel2d = kernel1d_y.unsqueeze(-1) * kernel1d_x
    return kernel2d

Direct link to code: https://github.com/pytorch/vision/blob/ad128b753c7e8cc0c600dfddac22ff48fc73c9d9/torchvision/prototype/transforms/functional/_misc.py#L85.

PIL's gaussian blur method takes in a single parameter, radius. I was wondering how to approximate Pytorch's gaussian blur using PIL?

radarhere commented 1 year ago

There's actually an open issue about this on pytorch - https://github.com/pytorch/vision/issues/5194

radarhere commented 1 year ago

pytorch's gaussian_blur_image_tensor takes a sigma argument.

https://github.com/python-pillow/Pillow/blob/ad938d54211c1cbded7ddb693cef4149e5674f20/src/libImaging/BoxBlur.c#L286-L290

Looking at https://www.mia.uni-saarland.de/Publications/gwosdek-ssvm11.pdf and the above code, sigma squared is equal to the radius squared divided by the number of passes (the number of passes is 3 by default).

Could you specify what you mean by 'approximate'?

github-actions[bot] commented 1 year ago

Closing this issue as no feedback has been received.