Closed pclucas14 closed 6 years ago
The previous version of SparseConvNet (https://github.com/btgraham/SparseConvNet) supported triangular/tetrahedral shaped filters.
Modifying Submanifold Sparse Convolutions to operate on the permutohedral lattice makes sense for working in 4+ dimensions, and fairly straightforward to implement. Given a point (x1,...,x{d+1}) in the permutohedral lattice, just store (x_1,...,x_d). For stride>1 up/down-sampling operations, regular SparseConvNet sparse convolutions could be used.
What is your use case?
My end goal is generative modelling of 3D data. I guess my main motivation is that the Splat / Convolve / Slice operations of 1 is appealing for generative modelling, since you can slice " the filtered signal onto a different set of points other than the input points". This then gives your a smooth / differentiable approach w.r.t to the (x,y,z) locations, which I think would be useful for point cloud generation.
From your comment, maybe the question I should be asking is "is there a way to perform a similar Splat / Convolve / Slice operation on a regular grid" ? We would then need to define barycentric interpolation for a square.
Hope this makes sense, Lucas
I am not planning on adding Splat/Slice operations in SparseConvNet, just Convolutions. A lot of what makes SparseConvNets efficient is the preservation of the sparsity pattern over many layers, which is incompatible with changing the sparsity pattern each layer with Splat/Slice operations. Regards Ben
Hi,
Do you think this implementation could be easily adapted to work with sparse permutohedral lattices ? As discussed in 1 2. I know both rely on efficient Hash Tables, so maybe there is an easy way to achieve this.
Thanks in advance, Lucas