search

traveller59 / spconv

Spatial Sparse Convolution Library
Apache License 2.0
1.87k stars 363 forks source link

Question about groups #416

Closed wencheng256 closed 2 years ago

wencheng256 commented 2 years ago

Thanks for your great work, I have some questions about the groups of convolution.

I notice that group operation is not implemented for now but I need to use this operation in my work. So I and my colleagues would like to implement it by ourselves and I am willing to submit our implementation to this repo if you like. To achieve this goal, I want to know:

Why it is not implemented? Are there any difficulties or just a lack of time? If there are some difficulties please tell me and we'll try to solve these problems.

Thank you

FindDefinition commented 2 years ago

The major reason is "group" isn't important in point-cloud networks:

  1. point cloud data is easier to learn than image
  2. the major bottleneck (speed) of sparse conv is IO, but the group conv can only save compute time. It's not hard to implement grouped sparse conv (gemm), but it's priority is low due to reasons above.
wencheng256 commented 2 years ago

Thank you very much

LayneH commented 2 years ago

Hi @wencheng256,

Have you implemented the Group Sparse Conv? I am also interested in this variant.

ruanych commented 2 years ago

The major reason is "group" isn't important in point-cloud networks:

  1. point cloud data is easier to learn than image
  2. the major bottleneck (speed) of sparse conv is IO, but the group conv can only save compute time. It's not hard to implement grouped sparse conv (gemm), but it's priority is low due to reasons above.

I'm also interested in support for grouping operations~

For the design of lightweight networks such as MobileNet, the use of depthwise separable convolutions (that is, setting the number of groups to the number of input channels) can reduce the amount of parameters.

For a kernel size of KxK, C channels (assuming the same number of input and output channels) parameters 2D 3D
convolution KxKxCxC KxKxKxCxC
depthwise separable convolution KxKxC + 1x1xCxC KxKxKxC + 1x1x1xCxC

The bottleneck of computing relative to IO may be broken by increasing the size of the convolution kernel. RepLKNet made an attempt, arXiv: https://arxiv.org/abs/2203.06717.

yxchng commented 4 months ago

any updates?