SparseConv2d VS torch.nn.Conv2d basic example

[calebtung]

Thank you for the great work!

Even after reading #142 , I'm still a bit confused; how would I implement a basic Conv2d operation on an RGB image using spconv?

Let's say in regular PyTorch, I did the following:

import torch

inputs = torch.random(4, 3, 400, 400) # batchsize of 4, 3 channel image (RGB), width/height: 400x400
torch_conv = torch.nn.Conv2d(3, 20, 3) # 3 input channels, 20 output channels, 3x3 kernel
outputs = torch_conv(inputs) # torch.Size([4, 20, 398, 398])

How would I achieve the same thing with spconv? Specifically, I'm confused about the expected arguments for features, indices, and spatial_shape.

I have this so far:

import torch
import spconv

inputs = torch.random(4, 3, 400, 400) # batchsize of 4, 3 channel image (RGB), width/height: 400x400
sparse_inputs = inputs.to_sparse()
spatial_shape = sparse_inputs.shape[1:-1] # that gives torch.Size([3, 400, 400]), is that correct?
batch_size = 4
features = sparse_inputs.values() # is this correct?
indices = sparse_inputs.indices() # is this correct?

sp_inputs = spconv.SparseConvTensor(features, indices, spatial_shape, batch_size)
sp_conv = spconv.SparseConv2d(3, 20, 3) # 3 input channels, 20 output channels, 3x3 kernel
sp_outputs = sp_conv(sp_inputs)

outputs = sp_outputs.dense(channels_first=False) # Also expect torch.Size([4, 20, 398, 398]), like 1st example

Any clarification on how to correctly implement this would be greatly appreciated; thank you!

[calebtung]

@traveller59 Would you be able to provide clarification on this? Thank you!

[sakura-iv]

Hello, if you want to get the same result, you should initialize the same weights and bias to them. I hope the code below would help you .

import numpy as np
import torch
import spconv

torch_tensor_dense = torch.rand(1, 4, 4, requires_grad = True)
torch_tensor_dense_to_sp = torch_tensor_dense.to_sparse() 

indices_dense = torch_tensor_dense_to_sp.indices().permute(1, 0).contiguous().int()
features_dense = torch_tensor_dense_to_sp.values().view(-1, 1)

sp_tensor_dense = spconv.SparseConvTensor(features_dense, indices_dense, torch_tensor_dense.shape[1:], batch_size = 1)

weight = torch.randn(1,1,3,3)
bias = torch.tensor([-0.05])

conv_nn_nopad = torch.nn.Conv2d(1,1,3,1,0)
conv_nn_nopad.bias = torch.nn.Parameter(bias)
conv_nn_nopad.weight = torch.nn.Parameter(weight)

conv_sp_nopad = spconv.SparseConv2d(1,1,3,1,0)
conv_sp_nopad.bias = torch.nn.Parameter(bias)
conv_sp_nopad.weight = torch.nn.Parameter(weight.reshape([3,3,1,1]))

nnconv_input_dense = torch_tensor_dense.unsqueeze(1)
out_nnconv_dense = conv_nn_nopad(nnconv_input_dense)
out_nnconv_dense

out_spconv_dense = conv_sp_nopad(sp_tensor_dense)
out_spconv_dense.dense()

the prints of out_nnconv_dense and out_spconv_dense.dense() are same.

[calebtung]

Thanks @sakura-iv, very helpful!

[FrancescoMandru]

Hey @calebtung @sakura-iv Did you solve your issue? What if I have 3D data? I would like to just use sparse convolution with a 3D point cloud

[jiaerwang0328]

@sakura-iv　Have you compared the inference time of the two convolutions?