search

zhijian-liu / torchprofile

A general and accurate MACs / FLOPs profiler for PyTorch models
https://pypi.org/project/torchprofile/
MIT License
560 stars 38 forks source link

[Feature Question]FLOPs computation support? #3

Closed isdanni closed 4 years ago

isdanni commented 4 years ago

I was testing different tools for computing FLOPs and MACs for CNN models and came across this repo. It seems like the FLOPs computation is not supported yet.

Was wondering if it's in future dev plan and if I could help with it? Thanks!

zhijian-liu commented 4 years ago

Hi, we choose MACs instead of FLOPs because it is more well defined. The computation of FLOPs has many choices to make. For instance, we need to decide how many FLOPs one division requires, one tanh requires, and one exp requires.

orashi commented 4 years ago

Yeah there are ambiguities in terms of how to deal with certain operations. Nevertheless, You may kindly just provide a verson of FLOPs stat practice of how your previous papers does it, It's been reviewed afte all. And that's the absolute mainstream metric of computational cost in the academic world.

isdanni commented 4 years ago

Yeah there are ambiguities in terms of how to deal with certain operations. Nevertheless, You may kindly just provide a verson of FLOPs stat practice of how your previous papers does it, It's been reviewed afte all. And that's the absolute mainstream metric of computational cost in the academic world.

Yes, actually I was wondering the same thing. Also, I wonder if we use a FLOPs computation tool(parameters all defined by users) to compare the performance of different models(e.g. resnet, vgg...), would that affect the result? I personally don't think so because the evaluation metrics are the same. (Please correct me if I'm wrong cuz I'm really new to this area)

So in that case, the accuracy of the FLOPs computation wouldn't be a problem?