search

NervanaSystems / neon

Intel® Nervana™ reference deep learning framework committed to best performance on all hardware
http://neon.nervanasys.com/docs/latest
Apache License 2.0
3.87k stars 811 forks source link

Compatibility of FC layers with Deconv Layers #144

Closed eracah closed 8 years ago

eracah commented 9 years ago

Is any way to connect an Affine layer to a Deconv layer? Like though a reshape layer or something? There seems to be issues with Affine layers not having a third dimension. Thanks!

ursk commented 9 years ago

Can you elaborate a bit what your use case it? Typically a deconv layer would take a feature map with some spatial dimension, and convolve that feature map with a filter. As you noticed, the affine layer has no concept of spatial dimensions in neon. In cases where we want to treat an affine layer as a convolutional layer with a spatial feature map of 1x1 pixels and the number of channels equal to the number of units in the layer, we do just that, make it a convolutional layer with feature map size 1x1. Would that work for your application?

eracah commented 9 years ago

Urs, I am making a convolutional autoencoder, where I have alternating conv and maxpool layers connected to a few FC layers and then those FC layers are upsampled with deconv layers. I wanted to have the FC layers, so I could have a learned feature vector that I could analyze. Great point about the 1x1xK feature map being equivalent to the output of connecting the last maxpool feature map to a FC layer of size K. I will implement that. Thank you for your help, Evan

scttl commented 8 years ago

Closing this as it looks like Urs' 1x1 convolution suggestion should meet your needs. Feel free to re-open if this isn't the case.