Does Flux support N-dimensional convolutions, where N>3?

[bhvieira]

While this works:

Conv((1,1,1), 1=>1)(rand(1,1,1,1,1)) #3D convolution

This throws an error:

Conv((1,1,1,1), 1=>1)(rand(1,1,1,1,1,1)) #4D convolution
#ERROR: MethodError: no method matching conv!(::Array{Float64,6}, ::Array{Float64,6}, ::Array{Float64,6}; pad=(0, 0, 0, 0), stride=(1, 1, 1, 1), dilation=(1, 1, 1, 1))

---

I'm running Julia 0.7 and:

[587475ba] Flux v0.6.7+ #master (https://github.com/FluxML/Flux.jl.git)
[872c559c] NNlib v0.4.2+ #master (https://github.com/FluxML/NNlib.jl.git)

[MikeInnes]

Not right now; each of 1-, 2- and 3-D convolutions just calls out to a specific kernel. But I'd happily take an implementation of an N-D convolution in NNlib, even if it's not heavily optimised.

[bhvieira]

@MikeInnes Right, thanks for the answer, I might look into it. I have 3D timeseries (fMRI volumes), and I did not want to go overboard with a 3D convolutional RNN, so I thought a simple 4D CNN (with kernel size 1 in the time dimension) could be a better idea. Do you want to keep the issue open as a feature request or should I close it?

[MikeInnes]

Yeah, we may as well keep this open, thanks.

[shreyas-kowshik]

I would like to work on this issue. Can someone please guide me as to what needs to be done?

[MikeInnes]

Might be best to speak to @avik-pal. On the GPU we're limited by what NVIDIA provides in CUDNN, but it should be straightforward to write an N-dimensional CPU kernel.

[avik-pal]

Unfortunately, CUDNN only provides 2D and 3D convolutions.

[shreyas-kowshik]

@avik-pal So should I go ahead with the CPU part? And if yes can you please guide me as to where should I start looking into the code?

[avik-pal]

@shreyas-kowshik you will first have to add it in NNlib. Here are the 2d and 3d implementations.

[datnamer]

Is it not possible to write a nd conv on the GPU?

[avik-pal]

@datnamer It is definitely possible. We need to write the CUDA kernel with CUDAnative and get it integrated with the Flux API. However, making the kernel efficient is not as straightforward as optimizing the CPU kernel.

[datnamer]

Understood

[shreyas-kowshik]

I was going through some code in NNlib.jl when I came across this : https://github.com/FluxML/NNlib.jl/pull/31

The PR effectively adds N-dimensional Conv to NNlib.jl

[cossio]

Should this be closed since https://github.com/FluxML/NNlib.jl/pull/94 has been merged?

[datnamer]

does that implementation work on GPUs?

[bhvieira]

@cossio it doesn't appear to work though. What should be the syntax to get (4+)D convolutions working?

[darsnack]

I think this should no longer be an issue.

[AriMKatz]

Even on CUDA?

[darsnack]

Not sure. Will test tomorrow morning (or someone else can race me).

[CarloLucibello]

On cpu, we support only up to 3d convolutions https://github.com/FluxML/NNlib.jl/blob/master/src/conv.jl I suspect it is the same for CUDNN but I didn't check

[darsnack]

Okay then we definitely should keep this open.