forward() got an unexpected keyword argument 'with_pooling'

[original-milk-tea]

Dear Dr.Du, thanks for your great job! I installed the environment according to the instructions in the readme, but when I ran python train.py, the following error occurred: ===> Refer-KITTI (train) <=== Number of identities: 475 ========== Training (Text-Guided OFF) ========== Traceback (most recent call last): File "train.py", line 88, in logits = model(inputs, epoch)['logits'] File "/home/zzz/miniconda3/envs/iKUN_Git/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/home/zzz/miniconda3/envs/iKUN_Git/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 171, in forward outputs = self.parallel_apply(replicas, inputs, kwargs) File "/home/zzz/miniconda3/envs/iKUN_Git/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 181, in parallel_apply return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)]) File "/home/zzz/miniconda3/envs/iKUN_Git/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 89, in parallel_apply output.reraise() File "/home/zzz/miniconda3/envs/iKUN_Git/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise raise exception TypeError: Caught TypeError in replica 0 on device 0. Original Traceback (most recent call last): File "/home/zzz/miniconda3/envs/iKUN_Git/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 64, in _worker output = module(*input, *kwargs) File "/home/zzz/miniconda3/envs/iKUN_Git/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "/home/zzz/Code/iKUN/model.py", line 267, in forward visual_feat = self.visual_local_global(x['local_img'], x['global_img']) File "/home/zzz/Code/iKUN/model.py", line 335, in visual_local_global local_feat = self.clip.visual(local_img, with_pooling=False) # [bt,c,7,7] File "/home/zzz/miniconda3/envs/iKUN_Git/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) TypeError: forward() got an unexpected keyword argument 'with_pooling' looking forward to your reply, thanks!

[original-milk-tea]

Actually, the self.clip.visual points to the VisionTransformer class. There is indeed no with_pooling parameter in the forward function of this class： Did I missed something?

[dyhBUPT]

Hi, thanks for your interest in our work! Sorry for the mistake, it seems a bug.

Indeed, we use ResNet50 for visual encoder, not ViT: https://github.com/dyhBUPT/iKUN/blob/dc8a529fd1897505bdd2fda13e9c75b317c9d766/opts.py#L26

And it seems that I modified the source code of CLIP, but I have forgot it. 😂😂😂 Specifically, I modified the function ModifiedResNet.forward() by adding the param with_pooling:

    def forward(self, x, with_pooling=True):
        def stem(x):
            for conv, bn in [(self.conv1, self.bn1), (self.conv2, self.bn2), (self.conv3, self.bn3)]:
                x = self.relu(bn(conv(x)))
            x = self.avgpool(x)
            return x

        x = x.type(self.conv1.weight.dtype)
        x = stem(x)
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)

        if with_pooling:
            x = self.attnpool(x)

        return x

Please let me know if this helps~

[original-milk-tea]

This really worked. Thanks for your kind help!

[hyman444]

(iKUN) PS D:\python\iKUN\iKUN> python test.py --test_ckpt iKUN.pth ========== Testing (Text-Guided OFF) ========== load from E:/data_base\ReferKITTI\iKUN.pth... ========== Testing Tracking ========== 0%| | 0/66456 [00:12<?, ?it/s] Traceback (most recent call last): File "test.py", line 230, in output = test_tracking(model, dataloader) File "test.py", line 145, in test_tracking similarity = model(inputs)['logits'].cpu() File "D:\Anacodna3\envs\iKUN\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "D:\Anacodna3\envs\iKUN\lib\site-packages\torch\nn\parallel\data_parallel.py", line 169, in forward return self.module(*inputs[0], *kwargs[0]) File "D:\Anacodna3\envs\iKUN\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "D:\python\iKUN\iKUN\model.py", line 267, in forward visual_feat = self.visual_local_global(x['local_img'], x['global_img']) File "D:\python\iKUN\iKUN\model.py", line 334, in visual_local_global local_feat = self.clip.visual(local_img, with_pooling=False) # [bt,c,7,7] File "D:\Anacodna3\envs\iKUN\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "D:\Anacodna3\envs\iKUN\lib\site-packages\clip\model.py", line 146, in forward x = stem(x) File "D:\Anacodna3\envs\iKUN\lib\site-packages\clip\model.py", line 141, in stem x = self.relu(bn(conv(x))) File "D:\Anacodna3\envs\iKUN\lib\site-packages\torch\nn\modules\module.py", line 1614, in getattr raise AttributeError("'{}' object has no attribute '{}'".format( AttributeError: 'ModifiedResNet' object has no attribute 'relu'

It can not run！

The original code was as follows. def forward(self, x, with_pooling=True): def stem(x): for conv, bn in [(self.conv1, self.bn1), (self.conv2, self.bn2), (self.conv3, self.bn3)]: x = self.relu(bn(conv(x))) x = self.avgpool(x) return x

    x = x.type(self.conv1.weight.dtype)
    x = stem(x)
    x = self.layer1(x)
    x = self.layer2(x)
    x = self.layer3(x)
    x = self.layer4(x)

    if with_pooling:
        x = self.attnpool(x)

    return x

Here are my modifications made under your guidance,can you help me to solve the problem above ?

def forward(self, x, with_pooling=True):
    def stem(x):
        for conv, bn in [(self.conv1, self.bn1), (self.conv2, self.bn2), (self.conv3, self.bn3)]:
            x = self.relu(bn(conv(x)))
        x = self.avgpool(x)
        return x

    x = x.type(self.conv1.weight.dtype)
    x = stem(x)
    x = self.layer1(x)
    x = self.layer2(x)
    x = self.layer3(x)
    x = self.layer4(x)

    if with_pooling:
        x = self.attnpool(x)

    return x

[hyman444]

I think it should be like this. def forward(self, x, with_pooling=True): def stem(x): for relu, conv, bn in [(self.relu1, self.conv1, self.bn1), (self.relu2, self.conv2, self.bn2), (self.relu3, self.conv3, self.bn3)]: x = relu(bn(conv(x))) x = self.avgpool(x) return x

[dyhBUPT]

Maybe that is because I use the old version of CLIP codes. Base on current codes, I think you are right. Just do it!

[dyhBUPT]

It's caused by this PR: https://github.com/openai/CLIP/pull/239