search

Haiyang-W / GiT

[ECCV2024 Oral🔥] Official Implementation of "GiT: Towards Generalist Vision Transformer through Universal Language Interface"
https://arxiv.org/abs/2403.09394
Apache License 2.0
301 stars 13 forks source link

FLOPs Calculation #15

Open OctoNinja9 opened 3 weeks ago

OctoNinja9 commented 3 weeks ago

Hi, @Haiyang-W! You have done a very interesting work. However, I encounter a problem when calculating the FLOPs of the GiT model. When I run python tools/analysis_tools/get_flops.py, it output 0 FLOPs.


loading annotations into memory...
Done (t=0.16s)
creating index...
index created!
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'BertTokenizer'. 
The class this function is called from is 'BlipTokenizer'.
10/08 15:06:10 - mmengine - WARNING - The following submodules of the model were never called during the trace of the graph. They may be unused, or they were accessed by direct calls to .forward() or via other python methods. In the latter case they will have zeros for statistics, though their statistics will still contribute to their parent calling module.
backbone, backbone.drop_after_pos, backbone.layers.0, backbone.layers.0.attn, backbone.layers.0.attn.proj, backbone.layers.0.attn.qkv, backbone.layers.0.ffn, backbone.layers.0.ffn.dropout_layer, backbone.layers.0.ffn.layers, backbone.layers.0.ffn.layers.0, backbone.layers.0.ffn.layers.0.0, backbone.layers.0.ffn.layers.0.1, backbone.layers.0.ffn.layers.0.2, backbone.layers.0.ffn.layers.1, backbone.layers.0.ffn.layers.2, backbone.layers.0.ln1, backbone.layers.0.ln2, backbone.layers.1, backbone.layers.1.attn, backbone.layers.1.attn.proj, backbone.layers.1.attn.qkv, backbone.layers.1.ffn, backbone.layers.1.ffn.dropout_layer, backbone.layers.1.ffn.layers, backbone.layers.1.ffn.layers.0, backbone.layers.1.ffn.layers.0.0, backbone.layers.1.ffn.layers.0.1, backbone.layers.1.ffn.layers.0.2, backbone.layers.1.ffn.layers.1, backbone.layers.1.ffn.layers.2, backbone.layers.1.ln1, backbone.layers.1.ln2, backbone.layers.10, backbone.layers.10.attn, backbone.layers.10.attn.proj, backbone.layers.10.attn.qkv, backbone.layers.10.ffn, backbone.layers.10.ffn.dropout_layer, backbone.layers.10.ffn.layers, backbone.layers.10.ffn.layers.0, backbone.layers.10.ffn.layers.0.0, backbone.layers.10.ffn.layers.0.1, backbone.layers.10.ffn.layers.0.2, backbone.layers.10.ffn.layers.1, backbone.layers.10.ffn.layers.2, backbone.layers.10.ln1, backbone.layers.10.ln2, backbone.layers.11, backbone.layers.11.attn, backbone.layers.11.attn.proj, backbone.layers.11.attn.qkv, backbone.layers.11.ffn, backbone.layers.11.ffn.dropout_layer, backbone.layers.11.ffn.layers, backbone.layers.11.ffn.layers.0, backbone.layers.11.ffn.layers.0.0, backbone.layers.11.ffn.layers.0.1, backbone.layers.11.ffn.layers.0.2, backbone.layers.11.ffn.layers.1, backbone.layers.11.ffn.layers.2, backbone.layers.11.ln1, backbone.layers.11.ln2, backbone.layers.12, backbone.layers.12.attn, backbone.layers.12.attn.proj, backbone.layers.12.attn.qkv, backbone.layers.12.ffn, backbone.layers.12.ffn.dropout_layer, backbone.layers.12.ffn.layers, backbone.layers.12.ffn.layers.0, backbone.layers.12.ffn.layers.0.0, backbone.layers.12.ffn.layers.0.1, backbone.layers.12.ffn.layers.0.2, backbone.layers.12.ffn.layers.1, backbone.layers.12.ffn.layers.2, backbone.layers.12.ln1, backbone.layers.12.ln2, backbone.layers.13, backbone.layers.13.attn, backbone.layers.13.attn.proj, backbone.layers.13.attn.qkv, backbone.layers.13.ffn, backbone.layers.13.ffn.dropout_layer, backbone.layers.13.ffn.layers, backbone.layers.13.ffn.layers.0, backbone.layers.13.ffn.layers.0.0, backbone.layers.13.ffn.layers.0.1, backbone.layers.13.ffn.layers.0.2, backbone.layers.13.ffn.layers.1, backbone.layers.13.ffn.layers.2, backbone.layers.13.ln1, backbone.layers.13.ln2, backbone.layers.14, backbone.layers.14.attn, backbone.layers.14.attn.proj, backbone.layers.14.attn.qkv, backbone.layers.14.ffn, backbone.layers.14.ffn.dropout_layer, backbone.layers.14.ffn.layers, backbone.layers.14.ffn.layers.0, backbone.layers.14.ffn.layers.0.0, backbone.layers.14.ffn.layers.0.1, backbone.layers.14.ffn.layers.0.2, backbone.layers.14.ffn.layers.1, backbone.layers.14.ffn.layers.2, backbone.layers.14.ln1, backbone.layers.14.ln2, backbone.layers.15, backbone.layers.15.attn, backbone.layers.15.attn.proj, backbone.layers.15.attn.qkv, backbone.layers.15.ffn, backbone.layers.15.ffn.dropout_layer, backbone.layers.15.ffn.layers, backbone.layers.15.ffn.layers.0, backbone.layers.15.ffn.layers.0.0, backbone.layers.15.ffn.layers.0.1, backbone.layers.15.ffn.layers.0.2, backbone.layers.15.ffn.layers.1, backbone.layers.15.ffn.layers.2, backbone.layers.15.ln1, backbone.layers.15.ln2, backbone.layers.16, backbone.layers.16.attn, backbone.layers.16.attn.proj, backbone.layers.16.attn.qkv, backbone.layers.16.ffn, backbone.layers.16.ffn.dropout_layer, backbone.layers.16.ffn.layers, backbone.layers.16.ffn.layers.0, backbone.layers.16.ffn.layers.0.0, backbone.layers.16.ffn.layers.0.1, backbone.layers.16.ffn.layers.0.2, backbone.layers.16.ffn.layers.1, backbone.layers.16.ffn.layers.2, backbone.layers.16.ln1, backbone.layers.16.ln2, backbone.layers.17, backbone.layers.17.attn, backbone.layers.17.attn.proj, backbone.layers.17.attn.qkv, backbone.layers.17.ffn, backbone.layers.17.ffn.dropout_layer, backbone.layers.17.ffn.layers, backbone.layers.17.ffn.layers.0, backbone.layers.17.ffn.layers.0.0, backbone.layers.17.ffn.layers.0.1, backbone.layers.17.ffn.layers.0.2, backbone.layers.17.ffn.layers.1, backbone.layers.17.ffn.layers.2, backbone.layers.17.ln1, backbone.layers.17.ln2, backbone.layers.2, backbone.layers.2.attn, backbone.layers.2.attn.proj, backbone.layers.2.attn.qkv, backbone.layers.2.ffn, backbone.layers.2.ffn.dropout_layer, backbone.layers.2.ffn.layers, backbone.layers.2.ffn.layers.0, backbone.layers.2.ffn.layers.0.0, backbone.layers.2.ffn.layers.0.1, backbone.layers.2.ffn.layers.0.2, backbone.layers.2.ffn.layers.1, backbone.layers.2.ffn.layers.2, backbone.layers.2.ln1, backbone.layers.2.ln2, backbone.layers.3, backbone.layers.3.attn, backbone.layers.3.attn.proj, backbone.layers.3.attn.qkv, backbone.layers.3.ffn, backbone.layers.3.ffn.dropout_layer, backbone.layers.3.ffn.layers, backbone.layers.3.ffn.layers.0, backbone.layers.3.ffn.layers.0.0, backbone.layers.3.ffn.layers.0.1, backbone.layers.3.ffn.layers.0.2, backbone.layers.3.ffn.layers.1, backbone.layers.3.ffn.layers.2, backbone.layers.3.ln1, backbone.layers.3.ln2, backbone.layers.4, backbone.layers.4.attn, backbone.layers.4.attn.proj, backbone.layers.4.attn.qkv, backbone.layers.4.ffn, backbone.layers.4.ffn.dropout_layer, backbone.layers.4.ffn.layers, backbone.layers.4.ffn.layers.0, backbone.layers.4.ffn.layers.0.0, backbone.layers.4.ffn.layers.0.1, backbone.layers.4.ffn.layers.0.2, backbone.layers.4.ffn.layers.1, backbone.layers.4.ffn.layers.2, backbone.layers.4.ln1, backbone.layers.4.ln2, backbone.layers.5, backbone.layers.5.attn, backbone.layers.5.attn.proj, backbone.layers.5.attn.qkv, backbone.layers.5.ffn, backbone.layers.5.ffn.dropout_layer, backbone.layers.5.ffn.layers, backbone.layers.5.ffn.layers.0, backbone.layers.5.ffn.layers.0.0, backbone.layers.5.ffn.layers.0.1, backbone.layers.5.ffn.layers.0.2, backbone.layers.5.ffn.layers.1, backbone.layers.5.ffn.layers.2, backbone.layers.5.ln1, backbone.layers.5.ln2, backbone.layers.6, backbone.layers.6.attn, backbone.layers.6.attn.proj, backbone.layers.6.attn.qkv, backbone.layers.6.ffn, backbone.layers.6.ffn.dropout_layer, backbone.layers.6.ffn.layers, backbone.layers.6.ffn.layers.0, backbone.layers.6.ffn.layers.0.0, backbone.layers.6.ffn.layers.0.1, backbone.layers.6.ffn.layers.0.2, backbone.layers.6.ffn.layers.1, backbone.layers.6.ffn.layers.2, backbone.layers.6.ln1, backbone.layers.6.ln2, backbone.layers.7, backbone.layers.7.attn, backbone.layers.7.attn.proj, backbone.layers.7.attn.qkv, backbone.layers.7.ffn, backbone.layers.7.ffn.dropout_layer, backbone.layers.7.ffn.layers, backbone.layers.7.ffn.layers.0, backbone.layers.7.ffn.layers.0.0, backbone.layers.7.ffn.layers.0.1, backbone.layers.7.ffn.layers.0.2, backbone.layers.7.ffn.layers.1, backbone.layers.7.ffn.layers.2, backbone.layers.7.ln1, backbone.layers.7.ln2, backbone.layers.8, backbone.layers.8.attn, backbone.layers.8.attn.proj, backbone.layers.8.attn.qkv, backbone.layers.8.ffn, backbone.layers.8.ffn.dropout_layer, backbone.layers.8.ffn.layers, backbone.layers.8.ffn.layers.0, backbone.layers.8.ffn.layers.0.0, backbone.layers.8.ffn.layers.0.1, backbone.layers.8.ffn.layers.0.2, backbone.layers.8.ffn.layers.1, backbone.layers.8.ffn.layers.2, backbone.layers.8.ln1, backbone.layers.8.ln2, backbone.layers.9, backbone.layers.9.attn, backbone.layers.9.attn.proj, backbone.layers.9.attn.qkv, backbone.layers.9.ffn, backbone.layers.9.ffn.dropout_layer, backbone.layers.9.ffn.layers, backbone.layers.9.ffn.layers.0, backbone.layers.9.ffn.layers.0.0, backbone.layers.9.ffn.layers.0.1, backbone.layers.9.ffn.layers.0.2, backbone.layers.9.ffn.layers.1, backbone.layers.9.ffn.layers.2, backbone.layers.9.ln1, backbone.layers.9.ln2, backbone.patch_embed, backbone.patch_embed.adaptive_padding, backbone.patch_embed.projection, data_preprocessor, embed, embed.LayerNorm, embed.dropout, embed.position_embeddings, embed.word_embeddings, synt_att, synt_att.out_proj, synt_position_embeddings
==============================
Use size divisor set input shape from (480, 640) to (1120, 1120)
==============================
Compute type: dataloader: load a picture from the dataset
Input shape: (1120, 1120)
Flops: 0
Params: 0.159G
==============================
Haiyang-W commented 1 week ago

I'm sorry, we haven't tested this part of the code. If I have time later, I will improve this part of the code.