search

SeungjunNah / DeepDeblur_release

Deep Multi-scale CNN for Dynamic Scene Deblurring
686 stars 143 forks source link

Measure the number of float-point-operations of the generator? #47

Closed hanzotruongdev closed 4 years ago

hanzotruongdev commented 4 years ago

I am finding the right way to measure/estimate the number of FLOPs of your model generator. As you know, there is some unofficial method for measuring the number of FLOPs of a model such as this one: https://github.com/apaszke/torch-opCounter. However, your model using muti inputs (3-time scales) so it is quite hard to measure the number of FLOPs. Could you please provide me a solution to measure FLOPs? I would appreciate it if you provide me some instructions, directions, code, or something like that. thank

SeungjunNah commented 4 years ago

Hi @noitq, I personally don't use the measuring tools for some reasons. 1) They are not guaranteed to measure the calculation with precision 2) When we use cudnn backend, FLOPS cannot easily be measured. The exact computation algorithm is not disclosed and we don't know what's going on inside the API. Note that conv algorithm can be changed without noticing when your memory status is different, especially when benchmark mode is turned on. I don't think the third-party tools can calculate unless they are provided by NVIDIA.

I would recommend calculating FLOPS by yourself, after deciding your target convolution algorithm. (naive, FFT, Winograd, etc.)

hanzotruongdev commented 4 years ago

I referred the paper DeblurGAN-v2: http://openaccess.thecvf.com/content_ICCV_2019/papers/Kupyn_DeblurGAN-v2_Deblurring_Orders-of-Magnitude_Faster_and_Better_ICCV_2019_paper.pdf They reported the FLOPs of your method (DeepDeblur). However, they didn't even mention the method for measuring.

Thank for your information @SeungjunNah, I have just known that there are so many implementations underlying the Convolution. Before that, I supposed that the Torch module uses a certain technique that I do not need to care about.

Anyway, I will dig more based on your keywords. Thank you so much!

SeungjunNah commented 4 years ago

The most popular convolution algorithm on GPU is Winograd. http://openaccess.thecvf.com/content_cvpr_2016/papers/Lavin_Fast_Algorithms_for_CVPR_2016_paper.pdf However, I believe most papers consider naive convolution to be standard when calculating FLOPS as GPU implementations heavily rely on parallelization. Other types of devices may perform differently.