Open PistonY opened 5 years ago
Hey, this is the MXNet Label Bot. Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it. Here are my recommended labels: Gluon, Feature
The Gflops is related to what kind of implementation is used and whether other optimization skills are applied.
For example, the computation of direct, GEMM-based and Winograd convolution is really different. And if the convolution is fused with BN/Relu/Sum, the calculation is also changed.
If you only care about how much OPs (Add, Mul, FMA) in the network at the runtime, I suggest to get it by profiling tools in a short time :)
Anyway, this is a good proposal for the analysis and I also like it.
@sandeep-krishnamurthy 's proposal maybe can cover your request with further improvements. https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks
@pengzhao-intel I mean GFLOPs is not consider special further improvements.It's just standard calculated amount. For example, a Conv2d flops should be defined as:
Assume that the input and output and kernel are square.
O(conv2d-nobias) = K * K * Cin * M * M * Cout
M = output feature map side length
K = kernel size side length
C(in/out) = Channels in/out
Thanks this will be very useful.
Currently, I am working on per operator profiling to capture forward time, backward time, max memory allocated in phase 1 of my work. And, these uses different input shapes and option for each operator.
Intention of this work is - to catch any performance regression at operator level, ability identify hot computation paths in operator kernel for certain input shapes, and finally, derive insights like - In MXNet ArgMax is slower than Max operator for same operation.
Will that give a proxy for what you are looking for? However, this proposal is more fine grained and will be useful for planning optimization work.
@sandeep-krishnamurthy It's pleasure this may help you, but I just want a static analysis as I reply to pengzhao-intel, that's not complicated as you do. The static analysis could help you evaluate network computing in the most fair way.
Anyone has an update on this ?
Now gluon has a
summary
function for calculate total params for a network but don't have a tool for calculate network FLOPs(G).Why need this?
But it's hard for calculating all of ops and actually not necessary(Some discussions here). So I partition them on demand.
Additional interface should be added to make users
defined calculations themselves
for manually defined Blocks.Welcome to add if anything missing. Welcome to suggest if anything wrong.