search

facebookresearch / fvcore

Collection of common code that's shared among different research projects in FAIR computer vision team.
Apache License 2.0
1.93k stars 226 forks source link

Does a single multiply op count one flop? #119

Closed jixiege closed 1 year ago

jixiege commented 1 year ago

flopcounter supports aten::upsample_nearest2d, but seems that this op does not involving multiply-add operations.

I am confused whether to strictly conform to the definition of MACs since "We count one fused multiply-add as one flop." is mentioned in docstring.

Does a single multiply count one flop?

ppwwyyxx commented 1 year ago

The idea was to count multiply-add as one MAC if the op use multiply-add, otherwise only count multiply so we don't ignore some possibly expensive ops. The flop counter provides APIs to overwrite these decisions.

This may not be ideal for a number of reasons, but that's mainly because MAC is a poorly defined concept (flop as well, but better).

I had attempted to change everything to flops instead of MAC in https://github.com/facebookresearch/fvcore/pull/77, but unfortunately could not get it approved at the time.

jixiege commented 1 year ago

The idea was to count multiply-add as one MAC if the op use multiply-add, otherwise only count multiply so we don't ignore some possibly expensive ops. The flop counter provides APIs to overwrite these decisions.

This may not be ideal for a number of reasons, but that's mainly because MAC is a poorly defined concept (flop as well, but better).

I had attempted to change everything to flops instead of MAC in #77, but unfortunately could not get it approved at the time.

Got it, thanks for your reply.