NVIDIA / MinkowskiEngine

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors
https://nvidia.github.io/MinkowskiEngine
Other
2.43k stars 360 forks source link

Asymmetric Convolution Kernel Size #553

Open xiaohao-lin1 opened 1 year ago

xiaohao-lin1 commented 1 year ago

Describe the bug For a non-regular "image", the same output size is given regardless of the kernel size.

A typical image is shaped (N, 3, H, W), but my input is multivariate timeseries signal shaped (N, 1, n_electrodes, n_timesteps) = (N, 3, 62, 1000). Where I tried to use Convolution of kernel sized (1,10) or (62, 1) to convolve along the time or electrode dimension respectively, I get the same sized output (N, 1, n_electrodes, n_timesteps), which equals to the input size.

This is not right.

kernel = (1,10) , expected output (N, 1, n_eletrodes, n_timsteps - kernel size + 1), actual output (N, 1, n_eletrodes, n_timsteps) kernel = (62,1) , expected output (N, 1, 1, n_timsteps), actual output (N, 1, n_eletrodes, n_timsteps)


To Reproduce

import torch import torch.nn as nn import numpy as np

from MinkowskiEngine import ( MinkowskiConvolution, MinkowskiELU, MinkowskiLinear, MinkowskiReLU, MinkowskiMaxPooling, MinkowskiBatchNorm, MinkowskiDropout,

) from MinkowskiOps import ( to_sparse, )

`class ToyCNN(nn.Module): def init(self): super(ToyCNN, self).init() self.conv1 = nn.Sequential( MinkowskiConvolution( in_channels=1, out_channels=1, kernel_size=10, stride=1, dilation=1,

has_bias=False,

            dimension=2),
    )

def forward(self, x):
    x_copy = x.clone()
    x_copy = x_copy.to(x.device)
    x_copy = to_sparse(x_copy)
    sparse = self.conv1(x_copy)
    dense = sparse.dense()[0] #s
    return dense `

class SpatialKernerl(ToyCNN): def init(self): super(SpatialKernerl, self).init() self.conv1 = nn.Sequential( MinkowskiConvolution( in_channels=1, out_channels=1, kernel_size=(62,1), stride=1, dilation=1,

has_bias=False,

            dimension=2),
    )

class TemporalKernerl(ToyCNN): def init(self): super(TemporalKernerl, self).init() self.conv1 = nn.Sequential( MinkowskiConvolution( in_channels=1, out_channels=1, kernel_size=(1,10), stride=1, dilation=1,

has_bias=False,

            dimension=2),
    )

def test_if_zeros_are_zeros(x): dash = '-'*20 print(f'{dash}test_if_zeros_are_zeros{dash}') print(x[:, :, :, 100:200] == 0) #100 zeros print(x[:, :, :, 300:350] == 0) #50 zeros print(x[:, :, :, 500:600] == 0) #100 zeros

print(f'{dash}test_for_non_zeros{dash}')
print(x[:, :, :, :100] == 0) #100 zeros

def tensor_to_cpu_np(x): return x.cpu().detach().numpy()

def get_models(): return [ TemporalKernerl(), SpatialKernerl(),

    TorchToyCNN(),
        ]

def loop_over_all_models(x): models_ls = get_models() for model in models_ls: model = model.to(device) y = model(x) print(f'-------------------------------------------{model.class.name}', y.shape) #torch.Size([2, 1, 62, 1000] y = tensor_to_cpu_np(y) test_if_zeros_are_zeros(y)

return y

if name == 'main':

models_ls = get_models()
batchsize = 2
in_channels = 1
nchannels = 62
ntimesteps = 1000
device = 'cuda:0'

x = torch.ones(batchsize, in_channels, nchannels, ntimesteps)
x[:, :, :, 100:200] = 0 #100 zeros
x[:, :, :, 300:350] = 0 #50 zeros
x[:, :, :, 500:600] = 0 #100 zeros

x = x.to(device)

loop_over_all_models(x)

``


Expected behavior A clear and concise description of what you expected to happen.


Desktop (please complete the following information):

wget -q https://raw.githubusercontent.com/NVIDIA/MinkowskiEngine/master/MinkowskiEngine/diagnostics.py ; python diagnostics.py

Additional context Add any other context about the problem here.