A question about the efficiency of 2D sparse operations

[lebron-2016]

Dear author,

When I was testing the performance of the model, I was confused about the efficiency of the 2D sparse operation, so I did the following experiment and found that 2D sparse convolution does not improve the efficiency compared to normal convolution. Could you please explain what the problem is? （version: spconv-cu113 2.3.3）

import spconv.pytorch as spconv
import torch
import torch.nn as nn
from spconv.pytorch import SparseConvTensor
import time

device='cuda:0'

x_d = torch.zeros((2, 4, 1024, 1024))
x_d[0,0,0:16,0:16] += 1.
x_d = x_d.to(device)
x = SparseConvTensor.from_dense(x_d.permute(0,2,3,1))

conv_sparse = spconv.SparseConv2d(4, 4, kernel_size=3,stride=2, padding=1,bias=False, dilation=1).to(device)
bn_sparse = nn.BatchNorm1d(4, momentum=0.1).to(device)
conv_bn_relu_sparse = spconv.SparseSequential(conv_sparse, bn_sparse, nn.ReLU(inplace=True)).to(device)

conv_norm = nn.Conv2d(4, 4, kernel_size=3,stride=2, padding=1,bias=False, dilation=1).to(device)
bn_norm = nn.BatchNorm2d(4, momentum=0.1).to(device)
conv_bn_relu_norm = nn.Sequential(conv_norm, bn_norm, nn.ReLU(inplace=True)).to(device)

start_time = time.time()
encoder_output1 = conv_bn_relu_norm(x_d)
part_time = time.time() - start_time
print("conv_bn_relu_norm time", part_time)
start_time = time.time()
encoder_output = conv_bn_relu_sparse(x)
part_time = time.time() - start_time
print("conv_bn_relu_sparse time", part_time)

start_time = time.time()
encoder_output1 = conv_bn_relu_norm(x_d)
part_time = time.time() - start_time
print("conv_bn_relu_norm time", part_time)
start_time = time.time()
encoder_output = conv_bn_relu_sparse(x)
part_time = time.time() - start_time
print("conv_bn_relu_sparse time", part_time)

start_time = time.time()
encoder_output1 = conv_bn_relu_norm(x_d)
part_time = time.time() - start_time
print("conv_bn_relu_norm time", part_time)
start_time = time.time()
encoder_output = conv_bn_relu_sparse(x)
part_time = time.time() - start_time
print("conv_bn_relu_sparse time", part_time)

Looking forward to your reply!

Thanks a lot!!

[lebron-2016]

Dear author,

I tried other versions of the spconv library and found that spconv-cu113 has obvious effects on 3D operations, but may be slower on 2D operations; however, some versions have obvious effects on 2D sparse convolution, but not so good on 3D. Have you encountered similar problems during your experiments?

Looking forward to your reply! Thank you for your help!!

[zhanggang001]

Dear author,

I tried other versions of the spconv library and found that spconv-cu113 has obvious effects on 3D operations, but may be slower on 2D operations; however, some versions have obvious effects on 2D sparse convolution, but not so good on 3D. Have you encountered similar problems during your experiments?

Looking forward to your reply! Thank you for your help!!

We encountered a similar problem. Unfortunately, we did not find the reason behind that yet. This may result from the implementation of the spconv library.