search

fredzzhang / upt

[CVPR'22] Official PyTorch implementation for paper "Efficient Two-Stage Detection of Human–Object Interactions with a Novel Unary–Pairwise Transformer"
https://fredzzhang.com/unary-pairwise-transformers
BSD 3-Clause "New" or "Revised" License
150 stars 24 forks source link

self.coop_layer or self.comp_layer #39

Closed chenminupup closed 2 years ago

chenminupup commented 2 years ago

Hello, may I ask why all the three indicators on Hicodet are 0.0000 after removing this layer？

fredzzhang commented 2 years ago

Hi @chenminupup,

I'm not sure what you mean. Are you referring to the mAP of the model? And what did you remove exactly, both layers or one of them?

Fred.

chenminupup commented 2 years ago

yes， I mean the mAP of the model is 0.0000 when I remove one or both of them。

The mAP is 0.0000, rare: 0.0000, none-rare: 0.0000

unary_tokens, unary_attn = self.coop_layer(unary_tokens, box_pair_spatial_reshaped)

pairwise_tokens, pairwise_attn = self.comp_layer(pairwise_tokens)

fredzzhang commented 2 years ago

Are you able to reproduce the results while keeping both layers?

chenminupup commented 2 years ago

yes，the results didn't differ much

fredzzhang commented 2 years ago

Hi @chenminupup,

I just repeated the experiment and got full: 0.2370, rare: 0.1701, none-rare: 0.2569 for UPT-R50 with both layers removed. I think you might have used an incorrect path to the checkpoint, in which case a randomly initialised model will be used, hence the zero mAP.

If the specified path is correct, you will see something like => Rank 0: continue from saved checkpoint checkpoints/sota/upt-r50-hicodet.pt on the command line, otherwise you will see => Rank 0: start from a randomly initialised model. Can you check if you've got the path correct?

Fred.

chenminupup commented 2 years ago

Hi， I have the correct file loading path, but the result is still 0.0000

=> Rank 0: continue from saved checkpoint /mnt/hdd1/cm/ckpt_upt/upt-r50-hicodet-no2att/ckpt_94080_20.pt 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9546/9546 [11:24<00:00, 13.94it/s] WARNING: Collected results are empty. Return zero AP for class 0. WARNING: Collected results are empty. Return zero AP for class 1. ... WARNING: Collected results are empty. Return zero AP for class 598. WARNING: Collected results are empty. Return zero AP for class 599. The mAP is 0.0000, rare: 0.0000, none-rare: 0.0000

fredzzhang commented 2 years ago

Based on the log, the network is returning empty results for each image. Can you attach the exact changes you made to remove the two layers?