apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.78k stars 6.79k forks source link

Identity mapping symbol in mxnet #3117

Closed VoVAllen closed 7 years ago

VoVAllen commented 8 years ago

Just want to dump some inner result in the computation graph

For example,

       net = mx.sym.Variable(name='X')
        net = mx.sym.Convolution(
                data=net, name='conv', kernel=(7, 7), num_filter=32)

        #dump the result after conv layer
        net=mx.sym.Identity(net)

        net = mx.sym.Activation(
                data=net, act_type='relu')
        net = mx.sym.Pooling(
                data=net, name='pool', pool_type='max', kernel=(2, 2),
                stride=(2, 2))
        net = mx.sym.Flatten(data=net)
        net = mx.sym.FullyConnected(
                data=net, name='fc1', num_hidden=hidden_size)
        net = mx.sym.Activation(
                data=net, act_type='relu')
        net = mx.sym.FullyConnected(
                data=net, name='fc2', num_hidden=num_classes)
        net = mx.sym.SoftmaxOutput(
                data=net, name='output')

In this example, I want to dump the image after conv layer. Because mxnet will only store the information about args, therefore I tried to manually add an argument to make the result stored by mxnet.

Any idea on how shall I get this identity mapping symbol?

piiswrong commented 8 years ago

dump to where?

VoVAllen commented 8 years ago

@piiswrong

Thank you for your reply. Dump to some variable. Actually what I want to do is to have the gradient of intermediate layer output (also the intermediate layer output), just like the data gradient as saliency map. Since the executor can have the grad of arg, therefore I want to take advantage of this feature.

piiswrong commented 8 years ago

You can group the intermediate layer output with the final output. You can also bind the gradient of intermediate layer's output

VoVAllen commented 8 years ago

great idea on group intermediate layer output with the final output! (I guess BlockGrad symbol is also needed here.) But I don't understand what do you mean by bind the gradient? It would be very helpful for me if you could provide a simple code example. Thank you.

phunterlau commented 7 years ago

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!