search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.77k stars 6.8k forks source link

Add FusedParam Support for Gluon LSTMs #14985

Open stephenrawls opened 5 years ago

stephenrawls commented 5 years ago

This is a feature request to have the Gluon LSTM operator use FusedParams for LSTMs and RNNs.

Here is the inference latency for the LSTM operator from the MKLDNN PR branch for a 1 layer 300dim lstm:

Total: 475us
    - rnn_param_concat: 250us
    - Reshape: 100us
    - RNN Op: 125us

As you can see a lot of overhead is spent concat'ing and reshaping individual params together into a fused param. Would be faster to just use fused params from the beginning.

mxnet-label-bot commented 5 years ago

Hey, this is the MXNet Label Bot. Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it. Here are my recommended labels: Gluon, Feature

vdantu commented 5 years ago

@mxnet-label-bot add [feature request, gluon]