search

ChenDelong1999 / ITRA

A codebase for flexible and efficient Image Text Representation Alignment
Other
16 stars 1 forks source link

ITRA冻结参数名称筛选bug #1

Open gzqy1026 opened 1 year ago

gzqy1026 commented 1 year ago

1、ITRA最近看了好像能正常运行，之前提的问题应该是我这边有问题 2、我觉得可能代码在冻住的部分逻辑有一些问题 image 如果想冻住resblocks1，代码判断逻辑是resblocks1是否在name里面，判断resblocks1在resblocks1(name)，那就成功冻住了。 但是resblocks1也在resblocks11(name)之中，所以冻住的层可能会判断出问题

我觉得可以这样，在resblocks1后面加上“.”判断“resblocks1.”是否在name里面，这样应该能区别“resblocks1.”和“resblocks11.”

ChenDelong1999 commented 1 year ago

https://github.com/ChenDelong1999/ITRA/blob/ee9f068497d4bfb3c8276b232b94e03519d10215/itra/training/model.py#L215-L243

ChenDelong1999 commented 1 year ago

指定 --lock-text-partial 'resblocks.1'确实会导致 resblocks.1，resblocks.10，resblocks.11都被冻住：

2023-01-12,15:38:01 | INFO | [main.py-262]: Prameters NOT to be optimized:
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.ln_1.weight   [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.ln_1.bias [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.attn.in_proj_weight   [1536, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.attn.in_proj_bias [1536]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.attn.out_proj.weight  [512, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.attn.out_proj.bias    [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.ln_2.weight   [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.ln_2.bias [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.mlp.c_fc.weight   [2048, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.mlp.c_fc.bias [2048]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.mlp.c_proj.weight [512, 2048]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.1.mlp.c_proj.bias   [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.ln_1.weight  [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.ln_1.bias    [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.attn.in_proj_weight  [1536, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.attn.in_proj_bias    [1536]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.attn.out_proj.weight [512, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.attn.out_proj.bias   [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.ln_2.weight  [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.ln_2.bias    [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.mlp.c_fc.weight  [2048, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.mlp.c_fc.bias    [2048]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.mlp.c_proj.weight    [512, 2048]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.10.mlp.c_proj.bias  [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.ln_1.weight  [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.ln_1.bias    [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.attn.in_proj_weight  [1536, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.attn.in_proj_bias    [1536]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.attn.out_proj.weight [512, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.attn.out_proj.bias   [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.ln_2.weight  [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.ln_2.bias    [512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.mlp.c_fc.weight  [2048, 512]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.mlp.c_fc.bias    [2048]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.mlp.c_proj.weight    [512, 2048]
2023-01-12,15:38:01 | INFO | [main.py-266]:     text_backbone.transformer.resblocks.11.mlp.c_proj.bias  [512]