How to measure the FLOPs of uniformer_large_ls

[yysung1123]

Hi,

I use the following code to count the FLOPs of Uniformer-L-LayerScale which is supposed to be 12.6G.

import torch
from fvcore.nn import FlopCountAnalysis
from fvcore.nn import flop_count_table
from token_labeling.tlt.models import uniformer_large_ls

model = uniformer_large_ls()
model.eval()
flops = FlopCountAnalysis(model, torch.rand(1, 3, 224, 224))
print(flop_count_table(flops))

But I got 19.7GFLOPs.

I would like to know how to measure it correctly. (get 12.6GFLOPs) Thank you.

[Andy1621]

@yysung1123 Very thanks for your question! You are right. The correct GFLOPs should be 19.8G, which is comparable with VOLO. I made a mistake because I used torch.utils.checkpoint, which ignored some layers for calculating GFLOPs. Sorry for the mistake and I will update the GFLOPs in the new version.

For the new comparison, the UniFormer-L seems not well-designed. From Base to Large, double GFLOPs but only 0.3 accuracy improvement on ImageNet. Due to the limited GPU resource, I did not scale up the model thoughtfully. I just used a wider channel to avoid Loss NAN in training. How to scale up a model better still need to be explored.

[yysung1123]

@Andy1621 Thanks for your replying.

The FLOPs of Uniformer-B is right (8.3G).

But I got 0.0% accuracy from the following script.

python3 validate.py ./imagenet   --model uniformer_base  --checkpoint ./uniformer_base_tl_224.pth --no-test-pool --img-size 224 --batch-size 128 --workers 8

uniformer_base_tl_224.pth is downloaded from ImageNet-1K pretrained with Token Labeling.

[Andy1621]

Can you provide a detailed log? It runs normally in my environment. Maybe you can check whether the model weight is loaded correctly.

[Andy1621]

As there is no more activity, I am closing the issue, don't hesitate to reopen it if necessary.