zackchase / mxnet-the-straight-dope

An interactive book on deep learning. Much easy, so MXNet. Wow. [Straight Dope is growing up] ---> Much of this content has been incorporated into the new Dive into Deep Learning Book available at https://d2l.ai/.
https://d2l.ai/
Apache License 2.0
2.57k stars 725 forks source link

Gluon example for multiple filters in CNN? #339

Open jdchoi77 opened 7 years ago

jdchoi77 commented 7 years ago

For the example code provided by https://mxnet.incubator.apache.org/tutorials/nlp/cnn.html

# create convolution + (max) pooling layer for each filter operation
filter_list=[3, 4, 5] # the size of filters to use
print 'convolution filters', filter_list

num_filter=100
pooled_outputs = []
for i, filter_size in enumerate(filter_list):
    convi = mx.sym.Convolution(data=conv_input, kernel=(filter_size, num_embed), num_filter=num_filter)
    relui = mx.sym.Activation(data=convi, act_type='relu')
    pooli = mx.sym.Pooling(data=relui, pool_type='max', kernel=(sentence_size - filter_size + 1, 1), stride=(1,1))
    pooled_outputs.append(pooli)

# combine all pooled outputs
total_filters = num_filter * len(filter_list)
concat = mx.sym.Concat(*pooled_outputs, dim=1)

# reshape for next layer
h_pool = mx.sym.Reshape(data=concat, target_shape=(batch_size, total_filters))

Could you please provide a code using Gluon that is equivalent to the above code? Would it be possible to accomplish this by using net = gluon.nn.Sequential() although those convolutions are performed in parallel for [3, 4, 5]-gram filters? Thank you!

zackchase commented 7 years ago

Is there a reason why you have to do this all in Sequential with primitive layers? Why don't you just write a block that does the multiple filters and then stack multiple such blocks with a sequential?

GaryGaryry commented 6 years ago

you can try gluon.Block https://mxnet.incubator.apache.org/api/python/gluon/gluon.html?highlight=block#mxnet.gluon.Block